Controlled Evaluation

A controlled evaluation is a test in which all conditions are deliberately kept the same and only a single thing is changed. The name says exactly that: “controlled” means kept under control, “evaluation” means assessment. An example from biology class: you want to know whether a fertilizer makes plants grow. So you place two groups of plants in the same light, water them equally often, and only one group receives fertilizer. If that group grows larger, it’s due to the fertilizer and not chance. Developers check whether a computer program has actually gotten better according to exactly this principle.

Why progress announcements are worth little without it

Companies constantly claim their new AI system is better than the old one. Without controlled conditions, such a claim is barely verifiable. Perhaps it was simply tested on different tasks. Perhaps the test ran on faster hardware. Perhaps the question was phrased slightly differently, which can already shift the result considerably.

Language models also don’t always answer the same way. The same question can lead to different answers twice in a row. This means a difference of one or two percentage points can be pure noise. A clean evaluation therefore repeats the test multiple times and states how much the results fluctuate. Only then can one say whether a lead truly exists.

This has tangible consequences. Companies decide which model to purchase for millions of euros based on such tests. Authorities rely on them when approving systems in healthcare or public administration. A poorly controlled test then leads not just to a false headline, but to a wrong decision.

What must stay the same and what is varied

At the outset there is a fixed collection of tasks, a so-called benchmark. This is a kind of exam for machines: always the same questions, always the same scoring. Both systems being tested receive exactly these tasks. The phrasing of the instructions, the order of the tasks, and the scoring rules also remain identical. Only the one point in question is changed, for example a new model or an altered setting.

An important point is the separation of the data. The test tasks must not appear in the material the model was trained on. Otherwise it recalls the answers from memory instead of working them out. Experts call this problem data contamination. It is the most common reason why impressive test scores fail to hold up in everyday practice.

Often, blind assessment is also used. People judging the answers do not learn which system they came from. This prevents a well-known brand name from influencing the grade. Related to this is A/B testing: there, two randomly formed user groups receive different versions of a piece of software. This too is a controlled evaluation, just with real users instead of a fixed list of tasks.

From model cards to the EU AI Act

Anyone reading the announcement of a new language model will almost always come across tables with percentage figures. These numbers stem from such tests. It’s worth looking at the fine print: does it state how often measurements were taken? Are margins of fluctuation given? Were the comparison models tested under the same conditions? If this information is missing, skepticism is warranted.

Regulation also picks up on this idea. The EU AI Act, Europe’s law on artificial intelligence, requires documented testing before market entry for high-risk applications. This concerns not only accuracy but also questions of fairness. For example, it is tested whether a system performs equally well across different population groups.

A common misconception is that a good test score automatically means good practice. A controlled evaluation only measures what is contained in the tasks. A model can excel at solving math problems and still fail at an ambiguously worded email. Control makes the result reliable, but not automatically meaningful for one’s own use case.