Full title: Plaintiff busted trying to use AI prompt injection to win court case, hides text instruction in filing — demands AI model reviewing the text should side with him, rumbled because of strange white spaces in text
Full title: Plaintiff busted trying to use AI prompt injection to win court case, hides text instruction in filing — demands AI model reviewing the text should side with him, rumbled because of strange white spaces in text
I feel like prompt injection should be a perfectly legal defense but only as long as it‘s phrased reasonably. Or is adding a note that asks for a fair trial unreasonable enough to be dismissed? When you only try to reason with unreasonable word salad how could anyone blame you for it?
Adding to that if one side uses LLMs they should definitely have to at least attach the prompts they used and share what model they used so it can be replicated.
An LLM’s response to a prompt is not necessarily a repeatable process.
I’m not aware of any public frontier LLM provider that uses a static seed for inference. Meaning, even with an identical prompt and identical model you will not get the same output. Seeds should absolutely come back with the streaming metadata on requests imho, but they don’t in any api/harness I’m aware of.
I know. But it needs to be replicated because an LLM can‘t actually explain itself. Because it doesn‘t understand words. If it can‘t be replicated or another session gives a completely different response that‘s their problem to deal with. Because if they legally try to treat LLMs as a consultant or expert, that expert should at least be consistent. If it can‘t well then you probably shouldn‘t use it blindly.