Chain-of-Thought Protocol

Programs like ChatGPT predict one word after another. They don’t have a notepad beside them on which they can silently calculate. If they spit out a number immediately, that’s more guessed than computed. That’s why they’re made to write out the solution path: first step one, then step two, and the result at the end. This very spelled-out intermediate reasoning is called a chain-of-thought protocol, in German roughly “Gedankenkette” (chain of thought). The text itself is the thinking, not merely a report about it.

Why writing it out raises the hit rate

A model has roughly the same amount of computation time available per generated word. A hard task therefore doesn’t automatically get more thinking time than an easy one. Forcing the model to write twenty intermediate steps gives it twenty portions of computation time instead of one. That’s the actual trick: the text is the working memory.

The effect is measurable. On math word problems, results in early studies rose from under 20 percent to over 50 percent, simply by prompting the model to proceed step by step. The gain is greatest on multi-step tasks: logic puzzles, programming bugs, legal cases. For a simple factual question, on the other hand, it brings almost nothing.

One distinction that often gets muddled is important. The protocol is not an honest self-report by the model. Studies show that models deliver answers that aren’t actually grounded in the written path at all. So the protocol can sound plausible and still bear no relation to the actual computational process.

From prompting trick to built-in procedure

At first, this was a matter of phrasing. You simply appended the sentence to the question that the model should proceed step by step. Alternatively, you showed it two or three example tasks along with a detailed solution path. The model would then adopt the pattern and, for the actual question, likewise write out a path.

Newer systems no longer need this prompt. They were specifically trained to internally generate a protocol before every answer. These are called reasoning models, that is, models with an outsourced computation step. They sometimes produce thousands of words of intermediate reasoning for a three-line answer.

An extension of this is called self-consistency. Here, the model is made to solve the same task multiple times, each time via a somewhat different path. Afterward, one counts which final result occurs most often. A calculation error is usually a one-off, while the correct path repeats. This, however, costs five to ten times the computation time.

Why providers hide the chain of thought

In most chat interfaces, you don’t see the protocol in full. Providers instead display a brief summary, often just a note like “thinking...”. The reason is economic: from complete protocols, competitors could retrain a model of their own. Some providers, for instance with openly available models, do show the chain in full.

In industry news, the term mainly comes up in connection with two topics. First, cost: long protocols mean many generated words, and billing is per token. A request to a reasoning model can therefore cost a multiple of a normal request. Second, safety: research teams read along in the protocols to detect whether a model is arriving at dangerous ideas.

One common misconception persists: many take the protocol to be a look into the model’s inner workings. It is not. It is generated text like any other, just text that happens to assist with computation. Anyone reading a justification should scrutinize it rather than treat it as proof.