Multi-Iteration Attack

Chat programs with artificial intelligence have rules about what they may and may not answer. Anyone wanting to bypass these rules can try a single blunt question. Usually this fails, because the system recognizes exactly such questions. A multi-iteration attack proceeds differently: the attacker poses many small questions one after another, each of which looks unsuspicious on its own. Only once combined do the answers add up to what was actually supposed to be blocked. The name says it all: the attack runs over several passes, that is, iterations, instead of in one go.

Why individual filters fail here

Most protection mechanisms check a request in isolation. They look at the current text and decide: allowed or not. This view is blind to a distributed attack. Because none of the individual requests crosses the line—only their sum does. A filter that evaluates each sentence on its own therefore never finds anything suspicious.

A comparison makes this clear. A store detective watches for someone stuffing an expensive item into their jacket. A thief who instead takes just one screw on each of twenty days doesn’t stand out. In the end, a whole device is still missing. The damage arises from the number of steps, not from the conspicuousness of any single one.

For companies this is a serious risk, because AI systems today see customer data, trigger orders, or search internal documents. If protective rules can be circumvented this way, data protection and liability are affected. Security researchers therefore count multi-iteration attacks among the hardest problems in AI applications. They cannot be solved by simply making the ban list longer.

The path through many small steps

Typical is the splitting of a forbidden task into innocuous sub-tasks. The attacker first asks for general background knowledge. Then they ask for clarification, then for an example, then for a technical detail. Each step is only a little further than the previous one. At no point does the model feel that it is crossing a line.

A second variant exploits the memory of the conversation. Language models factor the previous course of the conversation into their answers; this history is called context. Anyone who establishes a role early in the conversation, such as that of an examiner or author, can refer back to it later. The protective rules gradually lose weight as a result. Experts speak of crescendo attacks, because the pressure rises slowly.

Automated variants go even further. In this approach, a program tries out hundreds of phrasings and keeps the ones that get furthest. It slightly alters this phrasing and tests again. This procedure resembles breeding: what works gets reused. Sometimes a second AI model even takes on the role of the attacker.

Red teams, guardian models, and reports in the press

Large AI providers employ their own teams that attack their systems before others do. These groups are called red teams. Their reports regularly appear in the safety documentation for new models. When a provider writes there that a model has been hardened against multi-step circumvention attempts, this is exactly the topic being addressed.

In products, the countermeasure is usually invisible. An additional guardian model reads along with the entire conversation history and evaluates not the last question, but the direction of the conversation. Some providers also limit the number of requests per user per hour. Others reset conversations after a while so that no context can build up.

A common misconception is that a multi-iteration attack is simply an especially long jailbreak. The difference lies in the structure. In a classic jailbreak, the entire trick is contained in a single, often very cleverly constructed input. In a multi-iteration attack, no single input is problematic on its own, and that is exactly what makes defending against it so difficult.