Test-Time Reasoning

Programs like ChatGPT normally answer immediately. They generate one word after another and output the result directly. Test-Time Reasoning means: the program takes time first for difficult questions. It internally writes out intermediate steps, tries out paths, discards some again, and only answers afterward. “Test-Time” is the English term for the moment when the finished program is used — as opposed to the long learning phase before that. So part of the performance is shifted from learning to the moment of answering.

Thinking time as a second path to better answers

For years, a simple rule applied in the AI industry: if you want to be better, you have to build bigger. More training data, more data centers, more learning time. This path has become expensive, and progress has recently gotten smaller. Test-Time Reasoning opened up a second possibility in 2024. You simply let an existing program think longer and become better as a result, without retraining everything.

The effect is large for some tasks. In math competitions and coding, success rates rose in some cases from under 20 percent to over 80 percent. The reason is obvious: in a multi-step calculation, a single wrong intermediate step is enough to bring everything down. Anyone allowed to double-check and correct can still catch such errors themselves.

But there is a clear downside. An answer with a long thinking time can internally generate thousands of words that the user never sees. Nevertheless, every one of them is paid for. That’s why such answers are often ten to a hundred times more expensive and can take a minute instead of a second. For the question of the capital of France, this method is completely pointless.

What happens during thinking time

The basic technique is called chain of thought. The program formulates its solution path in sentences, the way one works through a word problem in writing. These intermediate steps are not incidental. They are part of the text the program is currently generating, and it can refer back to them for the next word. So the thinking is literally visible note-taking.

Several methods build on this. In one, the program generates several independent solution paths and takes the result that comes up most often — a kind of self-vote. In another, it checks its own intermediate solution, finds contradictions, and starts over. Some systems let a second, smaller model evaluate the steps and pursue the most promising ones further.

Importantly: the program does not gain any new knowledge in the process. It merely makes better use of what is already in it. A common misconception, therefore, is that long thinking prevents false claims. If a model doesn’t know a fact, it can make something up just as convincingly with five minutes of thinking time as it can in five seconds. The method is helpful where a solution can be verified.

Thinking models in current products

The breakthrough came in September 2024 with OpenAI’s model o1, followed by o3. Shortly after, the Chinese lab DeepSeek followed suit with R1 and released the weights openly, which caused a stir in the industry. Google offers Gemini with a “Thinking” mode, Anthropic offers “Extended Thinking” for Claude. All are based on the same principle.

In everyday use, such models can be recognized by a notice like “Thinking…” and a noticeable pause. Some providers display the chain of thought in an expandable field, others only summarize it. In developer interfaces, you can often set the thinking time yourself, for example in levels from low to high.

In stock market and business news, the term usually appears in connection with data centers. If every answer consumes more computing power, the demand for graphics chips rises not just for training but permanently for operation as well. This is a central argument in the debate about investments by Nvidia, Microsoft, and others.