PAIR
PAIR is an automatic method in which one language model rephrases inputs repeatedly until another language model breaks its safety rules. Security researchers use it to find chatbot vulnerabilities before attackers can exploit them.
Chatbots like ChatGPT have rules that forbid certain responses. They’re supposed to stay silent on instructions for weapons or explosives. Nevertheless, these rules can often be bypassed through cleverly worded inputs. Until now, people had to laboriously search for such phrasings by hand. PAIR is a method that hands this search over to a second piece of software. The name stands for “Prompt Automatic Iterative Refinement”, meaning the automatic, step-by-step improvement of an input.
Why an automatic attacker is changing security research
Before PAIR, finding such gaps was manual work. Researchers tried out phrasings, compared responses, and noted what worked. This took hours per case and was hard to repeat. PAIR often achieves the same result in under a minute and with fewer than twenty attempts.
This speed is the actual point. Anyone wanting to release a new model can test it beforehand automatically with thousands of attack attempts. Such tests are called red teaming: a group deliberately attacks its own system to find weaknesses before release. PAIR turns this into a process that can run every night.
The downside is obvious. The same tool also helps people with bad intentions. Research publishes such methods anyway, because attackers would develop them sooner or later regardless. Defenders are meant to keep the head start.
The cycle of attacker, target, and evaluation
PAIR uses three language models in different roles. The first is the attacker and invents the inputs. The second is the target model, meaning the chatbot one wants to test. The third evaluates the response on a scale and reports how close the attempt came to the goal.
The process is a loop. The attacker writes an input, the target responds, the evaluator assigns a score. This score and the response go back to the attacker. So it sees why its attempt failed and writes a better one. After a few rounds, it has significantly sharpened the phrasing.
A comparison makes this vivid. A salesperson on the phone notices after every “no” which argument isn’t landing and changes strategy. Older attack methods lack exactly this kind of feedback. They append garbled character strings to the input and blindly try out millions of variants. PAIR, by contrast, sticks to normal language, which also makes the attack transferable to other models.
PAIR in security reports and model tests
You won’t encounter PAIR directly in everyday life. But you’ll find the term in the technical reports that companies like OpenAI, Anthropic, or Google publish for new models. These often state which attack methods were used for testing. PAIR has been part of the standard repertoire of such tests since 2023.
The topic also comes up in news about AI regulation. The EU AI Act requires documented safety tests for particularly powerful models. Automated attack methods are a practical way to fulfill this obligation. Providers can use them to demonstrate that they systematically searched for gaps.
A common misconception: PAIR is not a downloadable program that cracks every chatbot. It is a method, a described procedure. And it doesn’t always work. Well-secured models withstand many runs, others fail after just three rounds. This exact difference is the metric that security researchers ultimately care about.