Effort Control

Modern chat programs like ChatGPT can handle a question with varying degrees of thoroughness. Sometimes they answer immediately, sometimes they first keep computing internally for quite a while before producing any output. Effort control is the setting that lets you specify exactly that. So you're not telling the program what to answer, but how much effort it's allowed to put in. Common levels are things like "low", "medium", and "high". A fitting comparison is a class test: given five minutes, you write down the first plausible solution; given an hour, you work through it and check the result.

Why thoroughness costs money

Every answer from an AI system costs computing time on expensive specialized chips. These costs don't occur just once, but with every single request. If a system keeps computing internally for a long time before answering, this effort multiplies. For a service with millions of users per day, that's a very large difference in the bill.

At the same time, by no means every question needs this thoroughness. "How many inhabitants does Hamburg have?" can be answered in one sentence. A multi-step physics problem or a programming bug in a thousand lines of code cannot. Without effort control, you'd have to settle for a compromise: either too expensive everywhere or too imprecise everywhere.

For companies, the dial is therefore above all a cost tool. They can use cheap levels for routine queries and reserve the expensive level only for cases where errors truly cause harm. Speed also plays a role: someone waiting for an answer in an app often finds twenty seconds of thinking pause worse than a slightly weaker answer.

What actually happens inside the model

So-called reasoning models write an internal chain of thought before the actual answer. This is text that the user usually doesn't get to see. In this text, the model breaks down the task, tries out intermediate steps, and discards approaches. Effort control mainly determines how long this internal part is allowed to become.

Technically, this is often regulated via a token budget. A token is a small unit of text, roughly a syllable or a short word. The "low" level might allow a few hundred such units for thinking, the "high" level several tens of thousands. Some providers additionally rely on the model having learned during training to adjust its pace to the given setting.

An important distinction: effort control does not make the model smarter. It doesn't unlock any additional capabilities, it just gives the same model more time. Another common misconception is that "high" is always better. For simple questions, prolonged deliberation sometimes causes the model to get tangled up in unnecessary edge cases.

The dial in apps and developer tools

In chat programs, effort control often appears as a model selection or as a toggle with names like "Think" or "Extended Thinking". You then choose between a fast and a thorough variant. Some providers also automatically decide, based on the question, which level fits.

Anyone accessing a model through a programming interface sees the dial more directly. At OpenAI, the parameter is called "reasoning_effort"; other providers let you specify a thinking budget as a number. Developers set this value per request, often depending on how difficult the task is judged to be.

In news about AI companies, the topic usually comes up in connection with costs and benchmarks. A model that performs especially well in a test has often used the most expensive effort level to do so. Comparisons between providers are therefore only fair if you know which setting was used for the measurement.