Persona-driven data synthesis

Programs like ChatGPT learn from vast amounts of text. Eventually, though, the usable material from the internet runs out or can no longer be used for legal reasons. In that case, the training material is generated by a computer program itself. With persona-driven data synthesis, the program is given a short role description each time: for instance, “a nurse working the night shift” or “a student who hates physics.” This produces very different texts from the same task, because each role phrases things differently, asks different questions, and chooses different examples. This diversity is precisely the purpose of the method.

The problem of uniformity

If you ask a language model a thousand times to invent a math problem, you get nearly the same thing a thousand times. It’s usually about apples, trains, or pocket money. Such texts are almost worthless as training material. A model trained only on this learns a narrow world and fails at everything outside it. Experts call this shrinking of training material mode collapse — the collapse of diversity.

Personas solve the problem at its root. If you ask the same program to phrase the task from the perspective of a logistics planner, it suddenly involves delivery times and inventory levels. A beekeeper delivers a problem about bee colonies. The content stays mathematics, but the packaging changes radically. With a million role descriptions, you thus get a million points of view.

This is economically relevant because good training material is expensive. Examples written by humans often cost several euros apiece. Synthetic examples cost fractions of a cent. Whoever can generate diversity cheaply saves tens of millions in the development of a model.

From role description to finished example

It starts with a collection of personas. These are usually derived from existing web texts: for each passage of text, a model is made to note what kind of person would write or read something like it. An article about soil samples then becomes, say, “an agricultural engineer testing fertilizers.” Afterwards, overly similar descriptions are filtered out so the collection stays genuinely broad.

Next, persona and task are combined into an instruction. Roughly: You are this person, write a question about this along with the solution. The model produces the text, and a verification step discards anything unusable. For math problems, the solution can even be checked automatically by computer. Whatever fails does not end up in the dataset.

One misunderstanding should be avoided: the personas are not real people and do not represent anyone. They are merely a trick to nudge the model’s randomness generator in ever new directions. You can think of them as a die with a million faces. The roll decides from which corner of the world the next example will come.

Where these datasets show up

The method becomes visible mainly in research reports and open datasets. It became well known through the Persona Hub project, which compiled around a billion role descriptions. Among other things, it was used to generate large quantities of math problems, coding examples, and dialogues. Several smaller models achieved results with it that previously only much larger systems could reach.

Indirectly, you encounter this method in every chat program that answers unusual specialist questions with surprising confidence. Some of this range does not come from real internet texts but from such invented examples. The idea is also used in software testing: hundreds of personas are made to try out the same application to find weaknesses.

In the news, this topic usually appears under the heading of synthetic data, often accompanied by warnings. If models are trained too long on nothing but their own output, their errors and biases get amplified. Personas mitigate this, but they do not eliminate it. That’s why developers generally mix synthetic and real data deliberately.