Generator-Evaluator Architecture

A computer program that generates text, images, or program code is constantly making decisions along the way. Some of them are good, some miss the mark. The generator-evaluator architecture is a design that defuses this problem. It consists of two separate roles: one role produces proposals, the other checks them and assigns a score. In the end, only the highest-scoring proposal is passed on to the user. The underlying idea is an everyday one: a text becomes better when someone else proofreads it than when the author sends it off unchecked.

Why checking is easier than inventing

For many tasks there is a clear gap between generating and checking. Solving a sudoku is laborious. Checking whether a filled-in solution is correct takes a minute. It is precisely this gap that the architecture exploits. The generating part is allowed to be uncertain, as long as the checking part recognizes the rejects.

Language models, meaning AI systems that continue text word by word, tend toward a well-known error: they invent plausible-sounding but false information. Experts call this hallucination. A single pass delivers this invention unfiltered. If a checking instance runs alongside it, however, an invented court ruling or a nonexistent source will, with a certain probability, be caught.

The procedure does come at a price, though. Generating and evaluating ten proposals requires significantly more computing time than a single one. That costs money and makes the response slower. Companies therefore deploy the architecture where errors are costly: in program code, medical summaries, or legal texts. For a casual chat question, the effort usually isn’t worth it.

The cycle of proposal and judgment

In the simplest case, the generator produces several variants of the same answer. This works because language models operate with an element of randomness and don’t always write the same thing given the same question. The evaluator gives each variant a score. The best one wins. This procedure is called best-of-n, where n stands for the number of attempts.

More sophisticated is the loop: the evaluator returns not just a grade but a justification. The generator then revises its draft accordingly. This repeats until the rating is good enough or a fixed number of rounds has been reached. Without this upper limit, both parts could spin in circles indefinitely.

Who provides the evaluator varies. Sometimes it is a second language model that reads and grades the answer. Sometimes it is a classic program without AI, such as a test run that checks whether written code starts without errors. Such hard checks are the most reliable, because they don’t guess themselves. If, on the other hand, a model is used as the checker, an important caveat applies: it can have the same blind spots as the generator and wave errors through.

From coding assistants to reasoning models

The principle is most visible in programming aids. Tools like GitHub Copilot or similar assistants write code, run automatic tests over it, and make corrections when a test fails. The user only sees the final result. The discarded attempts remain in the background.

The so-called reasoning models, which visibly think before answering, work in a related way. They generate several lines of reasoning and select the one that scores best. In technical articles, the term verifier appears for this, meaning checker. Reports of significantly improved results on math problems often trace back to such procedures.

A related but distinct concept is the GAN, the Generative Adversarial Network. There, a generator and a discriminator train each other up, and they do so during learning. The generator-evaluator architecture, by contrast, operates during live operation, with fully trained components. Confusing the two can easily lead to misclassifying news reports.