Evals

A program that writes texts or answers questions has to be checked somehow. You can't just claim it's good. So it's made to solve a collection of fixed tasks, and you count how many it answers correctly. Exactly these task collections plus grading rules are called evals, short for evaluations, meaning assessments. The principle is the same as with a classroom test: all test-takers get the same questions, and there's an answer key. The difference is that here it's not humans being tested, but software.

Why companies argue over test results

Evals are the only somewhat objective currency in the AI industry. When a company unveils a new model, it almost always shows a table of test scores. These numbers help decide who wins customers and who gets investor money. For developers, they're also the basis for selection: which model do we build into our product?

Evals are also indispensable internally. Anyone who changes something in a model wants to know whether it got better or worse. Without fixed tests, all that's left is a gut feeling, and gut feelings are deceptive. Improvements in one area often make another area worse. A model might get better at math after a change while simultaneously getting worse at summarizing.

At the same time, evals are controversial. A well-known problem is data contamination: if the test tasks are on the internet, they may end up in the training material. Then the model already knows the solutions and the test no longer measures anything. That's why some providers keep parts of their tasks secret.

From task sheet to percentage

An eval consists of three parts. First, the tasks, often several thousand. Second, the expected answers. Third, a rule that decides whether an answer counts as correct. In the end, there's usually a single percentage figure.

For multiple-choice questions, grading is simple: the letter is right or it isn't. For coding tasks, the generated code is run and checked against whether it passes the intended tests. It gets difficult with open-ended texts, such as a summary. Here there's no single correct solution.

For such cases, two approaches are used. Either humans grade the answers, which is thorough but slow and expensive. Or a second AI model takes on the role of grader. This method is called LLM-as-a-judge, meaning language model as judge. It's cheap and fast, but the judge has its own preferences. Longer and more politely phrased answers often score undeservedly well.

Well-known tests and a look at product announcements

In news about new AI models, the same names keep coming up. MMLU tests school and university knowledge across many subjects. GSM8K contains math problems at elementary school level that nevertheless require multiple steps. SWE-bench has models fix real bugs in real software. Anyone reading such a table should know: providers choose which tests to show.

A second venue is leaderboards, where people compare two anonymous answers and choose the better one. Millions of such comparisons produce a ranking, similar to the Elo rating in chess. This measures less knowledge than the impression an answer leaves on users.

Evaluation also happens outside research. Companies building a chatbot for their customer service create their own test sets with typical customer questions. Before every update, this test is run. A common mistake, by the way, is confusing high test scores with everyday usability. A model can brilliantly answer exam questions and still fail at the concrete task in actual operation.