Epoch Capabilities Index

The Epoch Capabilities Index is a metric that describes how capable an AI model is. It is calculated and regularly updated by the nonprofit research institute Epoch AI. The index combines results from various tests – so-called benchmarks, in which models must solve standardized tasks. These individual values are brought together into a single number that allows models to be directly compared. This makes it possible to see whether a new model is actually better than its predecessor – and by how much.

A common yardstick for a confusing landscape

Without such a metric, comparing AI models is difficult. Each manufacturer emphasizes different strengths of its model. One touts coding abilities, another language understanding, a third mathematics. This makes claims like “our model is the best” nearly impossible to verify.

The Epoch Capabilities Index provides a neutral basis here. Because it comes from an independent institute, it is less susceptible to deliberate embellishment. Journalists, researchers, and investors use it to put progress in AI development into perspective – for example, when assessing whether a newly announced model is truly a breakthrough or just a small step.

The index is also well suited to making trends visible over time. If you plot the values from recent years, you can see how sharply performance has increased. This is relevant for policymakers who want to estimate how quickly the technology is developing.

How the index is calculated

The foundation is benchmark results. A benchmark is a standardized test: for example, a model solves a hundred math problems, answers knowledge questions, or writes program code that is then automatically checked. Each task has a clear correct answer. This produces a measurable percentage value.

Epoch AI selects several such benchmarks that together cover a broad spectrum of capabilities. The individual results are weighted and combined into an overall score. It is important to note that not all benchmarks carry equal weight – some tasks are considered particularly indicative of real performance, others less so.

A well-known problem with such indices is so-called benchmark saturation: when models achieve nearly 100 percent on a test, differences can hardly be measured anymore. Epoch AI tries to counteract this by regularly adding new, more difficult tests to the selection. The index is therefore not a rigid tool but one that is continuously being developed further.

Where the index shows up in practice

In technology and financial media, the Epoch Capabilities Index appears mainly when a major new model is introduced. Editors and analysts use it to assess whether a model actually shifts the state of the art. An improvement of a few percentage points is usually considered incremental; a significant jump in the index suggests genuine progress.

The index is also cited in the political debate over AI regulation. Some regulatory proposals tie thresholds to performance measures – that is, the idea that beyond a certain level of model capability, special rules should apply. A uniform index makes it easier to define such thresholds.

For beginners, it is worth not viewing the index in isolation. It measures what models achieve on tests – not necessarily how useful they are in everyday life. A model can score highly on the index and still be disappointing in certain applications. The Epoch Capabilities Index is thus a useful tool for orientation, but not a verdict on usefulness.