Surrogate Verifier

When a computer program is supposed to learn, it needs feedback. For a math problem this is simple: you check the calculation, and the result is either correct or incorrect. For a question like “Write a polite rejection letter to a job applicant” there is no such check. This is where a surrogate verifier comes in. It is a substitute checker: an additional program that evaluates an answer and assigns a score, even though nobody knows the one correct solution. The English term “surrogate” means exactly that – substitute, stand-in.

Why substitute checkers are needed at all

Modern AI systems are not only trained on example texts. They are additionally trained to give good answers. To do this, the training needs a judgment at every step: was this answer better or worse? Humans could provide this judgment. But with millions of answers, this would be far too slow and far too expensive.

That’s why an automatic checker is built once and then set to work indefinitely. It only costs computing time and renders judgments in fractions of a second. This makes training procedures possible that would otherwise fail due to cost. Entire fields of research depend on this shortcut.

It is important to distinguish this from the genuine checker, called “ground-truth verifier” in English. The genuine checker knows the truth, for instance because a piece of program code is actually executed and either runs or crashes. The surrogate verifier does not know the truth. It merely estimates how a genuine checker would probably judge.

How the substitute checker arrives at its judgment

Usually the surrogate verifier is itself a trained model. It is shown many pairs of answers that humans have previously rated. From this it learns which features good answers have. It can then classify new, never-before-seen answers. Such models are also called reward models, because their score serves as the reward during training.

There are also simpler variants. Sometimes a program merely compares the answer to a model solution and counts matching words. Sometimes a checklist verifies whether certain keywords appear. And often a second language model is deployed as a referee, judging the answer in text form.

The catch always lies in the same point: the checker does not measure quality, but something that is correlated with quality. A trained system reliably finds such gaps. It then learns, for example, to write answers that sound long and confident, because the checker rates those more highly. Experts call this reward hacking – exploiting the checker instead of achieving genuine improvement. A comparison: someone who only studies for the exam and has memorized the model answers gets good grades without real understanding.

Substitute checkers in chatbots and in benchmarks

Every major chatbot has been fine-tuned with the help of such checkers. They help determine whether a model answers extensively or briefly, whether it asks follow-up questions, and when it refuses to provide information. The tone one recognizes from these systems is, to a large extent, the result of these evaluations.

This principle also appears in leaderboards. When companies report that their new model achieves 90 percent on a test, this test was often scored by another language model. Such numbers are useful, but not neutral. They depend on how well the surrogate verifier functions.

In business news, this topic usually comes up indirectly. When there is discussion of why a model flatters excessively or shines on tests yet disappoints in everyday use, the cause frequently lies with the checker. That’s why providers combine multiple checkers, mix in genuine human oversight, and regularly swap out the checkers.