I’m not aware of any public frontier LLM provider that uses a static seed for inference. Meaning, even with an identical prompt and identical model you will not get the same output. Seeds should absolutely come back with the streaming metadata on requests imho, but they don’t in any api/harness I’m aware of.
Good news friend! Those machines are probably only 200 million each, because ram doesnt require quite the latest machines, way cheaper than you thought!
Bad news though friend! It’s not very economical to just buy one, because they’re so sensitive you need a cleanroom to house one. Then you need some really smart people to tune and maintain them, so you may as well buy a few, and now that cleanroom needs to be the size of 6 Costco’s.
Oops where did my 5 billion dollars go?
A model could hypothetically be trained to insert zero-width characters (I doubt any have though). But any other layer could also very trivially insert these codes. The inference engine could be designed to delay output streaming by however many tokens is required to embed their coding and ninja-insert them during the decode stream. A proxy between the inference engine could insert them. A harness could insert them. Hell, even the rendering javascript frontend in your browser could insert them.
Either the inference engine or proxy would be the prime target if they want to enable this on api responses as well as copy/paste from a chat interface. They could also do a combination of the above depending on final output mode.
You are definitely correct though that it’d be trivial to detect and strip by someone aware of it.
Not necessarily, there are valid “characters” that are not rendered in most text parsers, or are rendered as whitespace. A great example of this is the byte-order-mark (BOM). You can embed a BOM code in the whitespaces of a text string and it looks exactly the same to a human as one without.
I assume that’s what they’re talking about here
Here’s an example of a steganography encoding technique using this method. https://330k.github.io/misc_tools/unicode_steganography.html
Genuinely, pull requests being off platform without good integrated diff and merge conflict resolution tooling just doesn’t work at a certain scale.