Pre-Generation Activations

A language model is a program that continues text by calculating, word by word, the most fitting next word. Before the first word of an answer appears, the program computes the question once in full. In doing so, vast quantities of intermediate numbers arise inside it. These intermediate numbers are called activations, and those that occur before the first output word are called pre-generation activations. They are something like the state of the model just before it speaks. Whoever reads them out does not see the answer, but the preparation for it.

What the numbers reveal before the first word

Asking a model for its reasons is unreliable. It then formulates an explanation that sounds plausible but does not necessarily describe what actually happened internally. The activations, by contrast, are not a narrative but the actual computational state. That is why they are considered a comparatively honest source.

The timing is especially interesting. If it is already possible to read off where the answer is heading before the first word, one can intervene before something problematic is output. Providers use this for safety filters: a small auxiliary program checks the activations and stops the response if it points in an undesired direction. This is faster and cheaper than checking the finished text afterward.

They are also valuable for research. One can test whether a model has actually understood a question correctly, or whether it already knows internally that it does not know the answer — and invents it anyway. Such cases are called hallucinations: fabricated statements presented with confidence.

From text to numerical state

The first step is called prefill. The model reads the entire input at once and sends it through its layers. A layer is a computational stage, and large models often have 40 to over 100 of them. In each layer, a long sequence of numbers arises for each word, frequently several thousand numbers long. These sequences are exactly what constitute the activations.

One can picture this like a factory with many assembly-line stations. The text goes in at the front, each station alters it somewhat, and only at the end does a word emerge. The pre-generation activations are what lies between the stations, just before the first piece leaves the factory.

To extract meaning from this, small auxiliary models, so-called probes, are trained. A probe is shown examples and learns to derive a simple statement from the numerical pattern. For instance: Is this request a deception attempt, yes or no? It is important to distinguish this from prompting: with prompting, one talks to the model; here, one measures it from the outside.

Where the term appears in reports

Most commonly, one reads about this in reports on AI safety. Labs such as Anthropic or OpenAI regularly publish studies in which they read out internal states in order to detect deception or dangerous intentions early. This field of research is called interpretability: the attempt to make a model’s inner workings legible.

A second place is products. When a chatbot rejects a request after mere fractions of a second, rather than first writing half a sentence, such a check is often behind it. Systems that automatically route requests to a cheaper or more expensive model also sometimes decide based on these early numerical patterns.

A common misconception is that the activations are something like thoughts. They are not. They are computational values without meaning in themselves; only a trained evaluation turns them into a statement, and that statement can be wrong. Still, the point is relevant for investors and observers: a provider that detects risks before output saves computing time and reduces its liability risk.