Verifier Model

Programs like ChatGPT generate text by guessing, word by word, what fits best. This often produces several possible answers to the same question. A verifier model is a second program that checks and evaluates these answers. So it doesn’t write anything of its own, but rather assigns something like grades. You can picture it like a proofreader sitting next to the writer. One produces suggestions, the other decides which of them is good enough.

Why checking is easier than solving

For many tasks, checking a solution is much easier than finding one. Solving a sudoku takes a long time, but checking the result takes a minute. A verifier model exploits exactly this gap. The generating model is free to produce twenty attempts, of which eighteen are wrong. As long as the verifier recognizes the two good ones, the overall result is strong.

This is especially important in mathematics, logic, and program code. There, a clearly correct solution exists, and errors stand out immediately. Research by OpenAI already showed in 2021 that math problems are solved correctly far more often with a verifier than without one. The trick wasn’t a bigger model, but more attempts plus good selection.

A second reason is safety. Verifiers can also check whether an answer violates rules, for example by containing dangerous instructions. Such checking models run in the background at many providers. They are usually smaller and cheaper than the actual language model, because evaluating requires less effort than generating.

From training dataset to grade

A verifier is trained with examples for which it is known whether the solution is correct. A language model is made to work through thousands of tasks, and each answer is marked as right or wrong. From these marked pairs, the verifier learns to recognize typical features of good and bad answers. In the end, it outputs a number for a new answer, for instance 0.87 as a confidence value.

There are two types to distinguish. An outcome verifier looks only at the final solution. A process verifier, on the other hand, evaluates every single computational step along the way. The second variant is more elaborate, because humans have to evaluate individual steps. In return, it catches errors earlier and is harder to trick.

One important special case doesn’t need a trained model at all. For program code, you can simply run the solution and let the tests execute. Such hard checks are more reliable than any estimate. Verifier models are used mainly where no automatic test exists, for example in word problems or explanations.

Verifiers in current reasoning systems

In the news, the term usually appears together with so-called reasoning models. These are systems that carry out longer intermediate deliberations before answering. They generate several solution paths and select the best one. It is often exactly this selection that a verifier takes over. Experts also call this procedure best-of-n: generate n attempts, keep the best one.

Verifiers also play a role in the training process itself. Their evaluations serve as a reward signal used to further improve the main model. In this context, they are often called reward models. The boundary between the two terms is blurry: a reward model rewards generally desired behavior, while a verifier specifically checks for correctness.

A common misconception is that a verifier guarantees correct answers. It is itself only a statistical model and can be wrong. If the generating system learns to specifically produce answers that please the verifier without actually being correct, this is called reward hacking. That’s why verifiers are, wherever possible, combined with real tests, such as executed code or recalculated formulas.