Activation

Modern AI systems consist of many small computational units called neurons. Each of these units collects numbers from the units before it, processes them, and passes on a single new number. This exact output number is called an activation. It indicates how strongly the unit in question responds to the current input. You can think of it like the volume at which a unit calls out its result into the next round. If the number is zero, the unit stays silent and contributes nothing.

What activations reveal about the inner workings of a model

An AI model stores its knowledge in fixed numbers called weights. These weights no longer change after training. Activations, on the other hand, are newly generated with every single request. They are thus what is currently happening, while the weights are what the model has learned.

That’s why researchers look closely at these intermediate values when they want to understand why a model produces a particular answer. An entire field of research is dedicated to mapping individual activation patterns to specific meanings. For example, patterns have been found that consistently become strong whenever a city is mentioned in the text. Such findings are the beginning of no longer treating AI systems merely as an inscrutable black box.

Activations also cost memory. While a model is computing, many of these intermediate values must be held simultaneously. With very long texts, this becomes the actual bottleneck, not the size of the model itself. Anyone wanting to reduce computing costs must therefore also keep an eye on activations.

From summing to the activation function

The path to an activation has two steps. First, the unit multiplies each input value by its weight and adds everything together. The result is a single number that can be arbitrarily large or small. This number is then passed through what is called the activation function.

This function bends the value according to a fixed rule. The most commonly used rule today is called ReLU and is remarkably simple: negative numbers become zero, positive numbers remain unchanged. Other functions instead squeeze the value into a range between zero and one. In every case, the result is the activation that gets passed on to the next layer.

This second step sounds like a minor detail, but it is crucial. Without it, the network would only add and multiply. Even with a hundred layers, it could then only represent simple, linear relationships. Only the bending introduced by the activation function makes it possible to capture complicated patterns like faces or sentence meanings.

The term in technical articles and tools

In reports on AI research, the word often appears in connection with safety. Companies like Anthropic or OpenAI publish studies in which they read out and interpret activations. Sometimes they even intervene actively and artificially amplify individual patterns. The model then behaves differently without having been retrained.

The term also comes up with image models. When a network processes a photo, the activations of the early layers can be displayed as an image. There you can recognize edges and color gradients. In deeper layers, the units respond to entire objects such as wheels or eyes.

A common mistake is confusing activations with parameters. When a model with seventy billion parameters is mentioned, this refers to the learned weights, not the activations. Parameters can be downloaded and stored on a hard drive. Activations only come into being the moment the model is actually computing, and disappear again afterward.