Verifier

A verifier is a checker. It receives a task and a proposed solution and states whether the solution is correct. The solution usually comes from an AI program that generates text or code. The verifier itself does not come up with anything, it only judges. This can be a fixed program, such as a calculator or a test run for program code. But it can also be a second AI model that has been specifically trained to detect errors.

Why checking is easier than solving

Behind the verifier lies a simple observation: finding a solution is often hard, but checking it is easy. Solving a Sudoku can take half an hour. Checking whether a completed Sudoku is correct takes a minute. This gap can be exploited. You let the AI generate many solutions and use the checker to select the best one.

This matters because language models sometimes make things up that sound convincing but are wrong. Experts call this hallucination. A model without oversight does not notice its own mistake. A verifier notices it, at least often. This turns a system that merely sounds plausible into a system that can be seriously deployed.

The benefit becomes especially clear in mathematics and programming. There, a clear right and wrong exists. With a poem interpretation it’s different — what counts as good depends on taste. That’s why verifiers work very well in some areas and barely at all in others.

From proposal to verified answer

The most common procedure is called best-of-n. The model generates, say, 64 different answers to the same question. The verifier assigns a score to each one. The answer with the highest score is output, the rest discarded. This works because of statistics: among 64 attempts, the correct solution is usually present, even though a single attempt often misses.

Two types are distinguished. A hard verifier checks mechanically and never errs: the code is executed, and either the tests pass or they don’t. A soft verifier is itself a trained model and only estimates how likely an answer is to be correct. It is more flexible, but it can be mistaken. Some systems check not just the final result but each computational step individually.

Verifiers are also used during training. They provide the signal by which the main model improves: correct solutions are rewarded, incorrect ones are not. A well-known risk here is called reward hacking. The model then learns to trick the checker instead of actually solving the task — for instance by writing code that passes exactly the test cases but is otherwise useless.

Verifiers in today’s AI products

When a chatbot briefly writes and runs Python code for a calculation task, a verifier is essentially at work there. The result of the calculation is objectively checkable, whereas a guessed result would not be. Programming assistants work similarly, automatically running their suggestion against existing tests before displaying it.

In news about AI, the term mostly appears in connection with so-called reasoning models, i.e. models that compute longer intermediate steps before answering. Much of their progress in math competitions is based on their training using verifiable tasks. Experts therefore say that AI improves especially quickly in areas where good verifiers exist.

This is exactly where the limitation follows. For tax advice, medical assessments, or journalistic texts, no automatic checker exists. There, the human remains the verifier. Anyone reading about AI progress can therefore always ask themselves one question: can it even be established mechanically in this domain whether the answer is correct?