Model Evaluation

A computer program that has learned from examples instead of being rigidly programmed is called a model. Such models answer questions, translate texts, or recognize objects in photos. No one can tell from the outside whether they do this well. So they are given test tasks for which the correct solution is already known, and the hits are counted. This exact process of checking is called model evaluation. The result is a number or a scorecard that can be used to compare different systems.

Why no one should deploy an untested model

An AI system often seems more convincing than it actually is. A chatbot formulates fluent sentences even when the content is wrong. The impression gained from trying it out is therefore regularly misleading. Only a measurement across many tasks shows how reliably the system really works.

For companies, a lot of money depends on this. Anyone buying software wants to know whether it is better than the existing solution. Developers themselves also need measurements. Without them, they could not tell whether a change to the model was an improvement or a deterioration.

On top of that comes the safety question. Models are also checked to see whether they issue dangerous instructions, disadvantage people, or can easily be tricked. The European AI Act explicitly requires such evidence for high-risk applications. Evaluation is therefore not just a technical matter, but also a legal obligation.

Test data, metrics, and benchmarks

The basic setup resembles a school exam. Before training, part of the data is set aside, the so-called test dataset. The model never sees these tasks during learning. Later it works on them, and its answers are compared with the known solutions. Had it already seen the tasks during training, it could simply reproduce them from memory.

The comparison produces metrics. The simplest is the accuracy: the proportion of correct answers. Often this is not enough. In a test for a rare disease, a model that always says 'healthy' would have an accuracy of over 99 percent and would still be worthless. That is why additional measurements are taken of how many real cases are found and how many alarms are false alarms.

Standardized collections of tasks are called benchmarks. Well-known examples contain exam questions from school and university, or programming tasks in which the written code is automatically tested. For open-ended texts, there is no clear-cut model answer. In that case, either humans evaluate the answers, or a second, strong model takes on the role of examiner.

Leaderboards in the headlines and their pitfalls

At every unveiling of a new AI model, providers show bar charts with benchmark results. Headlines like 'Model X beats the competition in eight out of ten tests' come from exactly this. Public leaderboards, in which users compare two anonymous answers and pick the better one, also belong to this category. Stock prices of technology companies sometimes react to such figures.

Caution is nevertheless warranted. Many benchmark tasks are freely available on the internet and thus accidentally end up in the training data. The model then already knows the answers, and the score is worthless. Experts call this problem data contamination.

A second misconception is widespread: a high score on a test does not mean the model is suitable for your task. Companies therefore build their own small test collections from real cases out of their everyday operations. This in-house testing often reveals more than any public leaderboard.