Polysemanticity

In an AI language model there are millions of small computational units called neurons. One might expect that each neuron takes on exactly one task — for example, always activating when the model processes the word “dog.” In practice, this is usually not the case. A single neuron often fires in response to completely different concepts: to “dog,” to “running,” and to “autumn” — without any discernible connection. This behavior is called polysemanticity. The term comes from linguistics, where it describes the fact that a word can have multiple meanings — here it has been applied to model units.

Why polysemanticity makes AI research more difficult

Anyone who wants to understand how an AI model makes decisions looks at which neurons are active for a given input. But this only works if each neuron has a clear role. If a neuron is polysemantic — that is, responsible for multiple concepts at once — it becomes nearly impossible to say why it is active in a particular case.

This is a particular problem for AI safety research. There, the goal is to determine whether a model is processing dangerous or undesirable concepts. Polysemanticity makes this kind of oversight considerably harder. You can see that something is happening — but not what.

A common misconception is to assume that polysemanticity is a flaw that could have been avoided during training. In fact, it is considered an almost unavoidable consequence of models having to fit a very large number of concepts into a limited number of neurons. The neurons, so to speak, share the available space.

Superposition: the reason for the overlap

Researchers typically explain polysemanticity using the principle of superposition. A model has to represent far more concepts than it has neurons. The solution: multiple concepts are stored overlapping within the same neurons — similar to how many voices can be recorded simultaneously on a single audio track. This saves space, but makes the signal difficult to decode.

The consequence is that polysemanticity is not a coincidence. Models actively learn to stack concepts, because it is the only way to make do with the available resources. Smaller models do this more than larger ones, because their lack of space is more severe. Larger models therefore tend to be easier to analyze — but they are not free of the phenomenon either.

A newer research approach called Sparse Autoencoders — that is, sparse decomposition methods — attempts to untangle polysemantic neurons. The idea: a second, small network is trained to rewrite the activations of the actual model so that each unit once again stands for only one concept. The results are promising, but the method is still in the research stage.

Polysemanticity in the current AI debate

The topic is especially prominent in a research field called Mechanistic Interpretability. The goal is to understand AI models from the inside, rather than merely testing them from the outside. Polysemanticity is one of the core problems that must be solved before further progress can be made.

In 2023, the AI company Anthropic published a widely noted study in which it examined millions of neurons in its Claude model and actually managed to isolate monosemantic — that is, clearly assigned — units. This was regarded as a breakthrough, because it showed that polysemanticity can at least partially be resolved.

In news articles, the term comes up whenever the discussion turns to AI transparency or the question of whether models can be trusted. Anyone seeking to understand why AI systems are so difficult to control will sooner or later encounter polysemanticity as one of the technical roots of this problem.