Activation Space

An AI system does not compute with words, but exclusively with numbers. When you enter a sentence, it is immediately translated into long lists of numbers. These lists travel through many computational layers, changing step by step along the way. Each such list of numbers is a point in an imagined space, and this exact space is called the activation space. You can picture it like a map on which the system deposits every intermediate stage of its processing. However, the space does not have two or three dimensions like a real map, but often several thousand.

Why meaning becomes proximity there

The astonishing thing about this space is that closeness within it means something. Points that belong together in content lie close to one another. The number lists for “dog” and “cat” are more similar to each other than those for “dog” and “tax return.” No one programmed this in on purpose. This order emerges by itself because the model sees millions of examples during training and stores similar relationships in similar ways.

This yields a practical benefit. When meaning becomes measurable as distance, similarity can be calculated instead of guessed. A search engine can thus find texts that match a query even if not a single word matches literally. Recommendation systems work on the same principle with movies, products, or music.

Even more important is the activation space for research into the safety of AI. Large models are barely comprehensible to humans from the outside. Anyone who wants to know why a system gives a particular answer has to look inside. The activation space is the place where this interior becomes visible in the first place.

Directions, layers, and the view inside

Technically, a model consists of many layers connected one after another. Each layer takes in the numbers from the previous one, computes with them, and passes on new numbers. These output values are called activations. Each layer therefore has its own activation space, and these spaces differ greatly from one another.

In early layers, the numbers still stay close to the pure word form. Further along, they become more abstract and tend to capture meaning, tone, or intent instead. Researchers extract the numbers at a particular point and examine them using statistical methods. One goal here is to find individual directions in the space that stand for a clear concept.

Such directions can not only be read but also changed. If you shift the activations along a direction that has been found, the model’s behavior changes. In this way, systems have been made to answer more politely or even more riskily, without retraining them. This targeted intervention is called steering, and it is currently a very active field of research.

From vector search to lie detector

You most commonly encounter the activation space in the form of so-called embeddings. These are lists of numbers that a model outputs for a text or an image and that are stored permanently. Companies place millions of these in vector databases. When someone asks something, the system searches for the nearest points and delivers the corresponding documents. This is exactly how many chatbots work that provide information about internal company documents.

In trade news, the term also comes up in connection with interpretability. Labs such as Anthropic or OpenAI regularly publish papers on which concepts they have found in the activation space of their models. Some approaches even check whether a model internally represents something different from what it outputs. Such a lie detector is not reliable so far, but the intent behind it is serious.

One common misconception should still be cleared up. The activation space is not the model’s memory, nor is it its stored knowledge. The knowledge resides in the weights, that is, the fixed numbers learned during training. The activations, by contrast, arise anew with each individual request and disappear again afterward. It can be compared to the difference between the school material you have learned and your thoughts during a specific task.