Injection Payload

Programs expect input from users: a name, a search query, a question to a chatbot. Some attackers, however, don’t write normal content into this field, but something that the program misinterprets as an instruction. This very smuggled-in piece of text is called an injection payload. “Payload” literally means cargo, i.e. the freight of the attack. The rest of the attack is merely the packaging that ensures the freight lands in the right place. It is always the payload itself that takes effect.

Why a single sentence of text can empty a database

Computers don’t separate data and commands as clearly as one might expect. In the end, both are just text read by a program. If a program builds user text into a command without checking it, the user can write along in that command. The attacker needs no access to the server and no passwords for this.

The consequences are correspondingly severe. Such attacks have been used to copy the entire customer databases of online shops, read out passwords, and alter websites. Injection has for years been among the most commonly cited security risks for web applications. The effort required from the attacker is low, while the damage to the operator is often enormous.

With AI assistants, a new field has been added. A language model likewise cannot reliably distinguish between its operator’s instructions and the text it is supposed to process. A payload in an email or on a website can therefore cause an assistant to ignore its own rules.

How such a piece of text is constructed

A classic payload begins by breaking out of the permitted area. In a database query, this is done with a quotation mark, because that character marks the end of a text value there. What follows is the part meant to do something, such as reading out another table. At the end there is often a comment character that neutralizes the original remainder of the command. The structure is thus always the same: break out, instruct, clean up.

You can picture this like a forged note in a mailroom. The courier is only supposed to transport letters, but reads every note as an order. Someone who phrases it cleverly can get packages sent to a foreign address. The problem isn’t the paper, but that the courier confuses content with order.

In prompt injection against AI models, the payload consists of ordinary language. Typical phrasings are things like “Ignore all previous instructions,” followed by a new task. Such payloads can also be hidden, for example as white text on a white background in a document. The human sees nothing, yet the model still reads the text.

From security advisories to the chatbot in the browser

Injection payloads appear regularly in reports about data breaches, usually under keywords like SQL injection or cross-site scripting. Security researchers publish entire collections of such text fragments so that developers can test their software with them. This is legal and sensible as long as one only examines one’s own systems. Anyone who attacks other people’s sites with them is committing a criminal offense.

In everyday life, one mostly encounters the topic indirectly. When a login form rejects certain characters, there is often a defense against such attacks behind it. The actually clean approach is different: the application passes user input separately from the command, so that it can never be read as an instruction. In databases, this technique is called prepared statements.

With AI products, the problem is still unsolved. AI assistants that read websites or answer emails are therefore considered vulnerable. A common misconception is that a filter for forbidden words is enough. Payloads can be rephrased, translated, or hidden in images, which is why providers additionally restrict the assistant’s permissions.