Scratchpad

Programs like ChatGPT write text word by word. They have no hidden memory in which they could calculate at their leisure. Everything they “think” they must actually write down. A scratchpad exists exactly for this: a note-taking area where the program records intermediate steps before stating the final answer. The name is meant literally, since “scratch pad” refers to a pad for jotting down notes. So instead of immediately spitting out a number, the program first notes down the calculation path.

Why a scratch pad lowers the error rate

For each individual word, the program only has a limited amount of computing time available. Solving a difficult task in a single step is therefore often impossible. If the program instead writes twenty intermediate steps, it has twenty times as much computing time available. The scratch pad literally buys the model thinking time.

There is also a second effect. The program can re-read every word it has already written when producing the next word. An intermediate result that has once been put on the pad no longer needs to be held in mind. The text itself becomes the memory. Without notes, the model would have to process all partial results simultaneously, which quickly goes wrong on longer tasks.

The difference is measurably substantial. On math word problems, success rates through step-by-step calculation often rise from under thirty to over sixty percent. The model has not become smarter in the process. It is merely making better use of the abilities it already had.

From prompt to built-in thinking area

The simplest variant is a sentence in the input: “Think step by step.” This instruction is called a prompt, i.e., a directive to the model. It is often enough on its own to make the program lay out its calculation path instead of guessing. One can also supply a solved example for the model to orient itself by.

The second variant is more elaborate: the model is specifically trained to produce intermediate steps. To do this, it is presented with thousands of tasks along with their written-out solution paths during training. Such models then note things down on their own, without being asked to. In newer systems, this note-taking area is permanently built in and is no longer even fully shown to the user.

A common misconception is that the scratchpad is a record of the actual internal processes. That is not true. The notes are themselves merely generated text and can be wrong even while the final result is correct, or vice versa. A model can write down a flawed calculation path and still arrive at the right answer by chance. So one should read the pad, but not blindly trust it.

Where the notepad shows up in products

When a chatbot provides a numbered list of considerations for a puzzle task, you are seeing a visible scratchpad. With so-called reasoning models, i.e., models with a built-in thinking phase, often only a note like “thinking” appears instead. The notes then run in the background and are discarded or shortened before the answer.

This comes at a price that regularly shows up in business news. Every word on the scratch pad costs computing time and therefore money, even though the user never gets to see it. Providers bill for these invisible words. A model that thinks for a long time can cost a multiple of the price for the same question.

The term also plays a role in the topic of safety. Researchers read along in the notes to detect whether a model is arriving at its answer via a sensible path. At the same time, there is growing concern that models might be planning something different in their note-taking area than what they ultimately say. You will keep encountering both debates in news around AI.