Capability Extraction

An AI system like ChatGPT has derived patterns from huge amounts of text during a long learning phase. What capabilities emerged from this, nobody knows exactly at first — not even the developers. Capability Extraction is the umbrella term for all techniques used to make this hidden capability visible and usable. The model itself is not changed in the process; only the way it is queried or used is changed. A comparison: a student might answer a task incorrectly in their head, but solve it correctly once they work through it step by step in writing. The knowledge was already there beforehand, only the way of asking was clumsy.

Why nobody knows what a model can really do

When a company releases a new model, it wants to measure its capabilities. For this purpose there are standardized test series, so-called benchmarks, such as math problems or programming tasks. The problem: the result depends heavily on how you ask. The same model can achieve 40 percent or 80 percent on the same set of tasks, depending on the querying method. A single number therefore says little.

This becomes especially important in safety testing. Before a powerful model is released, review teams test whether it reveals dangerous knowledge — such as instructions for weapons or for attacks on computer systems. If these reviewers only ask superficially, the model appears harmless. An attacker with better methods may later extract what is actually inside it after all. That is why safety teams deliberately try to extract as much capability as possible before others do.

A second reason is economic in nature. Training a new model often costs hundreds of millions. Better querying methods for an existing model cost almost nothing. Anyone who manages to extract another ten percent of performance from an existing system may save themselves an entire training run.

The tools of coaxing it out

The simplest technique is skillful formulation of the request. You ask the model to think out loud and write out the calculation step by step. This method is called Chain of Thought. On math problems, the success rate can rise dramatically as a result. Also effective: prefacing the model with two or three solved example tasks so that it recognizes the desired answer format.

A second group of methods makes use of repetition. You ask the same question twenty times and let the model generate twenty solution paths. You then choose the answer that occurs most often, or have a checking program test the suggestions. This works well for programming, because code can simply be run and checked for errors.

In safety testing, a third group is added: the model is lightly fine-tuned with a few examples in order to bypass built-in restrictions. Experts call this fine-tuning. This allows review teams to check whether a dangerous capability is truly absent or merely suppressed. The difference is crucial, because suppressed knowledge can be brought back, whereas absent knowledge cannot.

Where the term appears in reports and debates

Capability Extraction is most often mentioned in the safety reports that providers such as OpenAI, Anthropic, or Google publish for new models. There it typically states, in effect, that considerable efforts were made to fully draw out the model’s capabilities. This statement is important for regulators, because a weak test creates false security.

The topic also plays a role in rankings. When two models are compared, experts regularly argue over whether both were queried equally well. A model with a worse querying method performs unfairly poorly. That is why serious comparisons now also state which method was used.

A common misconception is to confuse Capability Extraction with training new capabilities. The model does not learn anything new here. One merely finds better access to what is already present. Understanding this distinction also helps explain why a model, months after its release, suddenly solves tasks it initially failed at.