Attention Pattern

When a computer program processes a sentence, it does not look at the words individually and independently. For each word, it also looks at the other words in the sentence — some strongly, some barely at all. This distribution is called an attention pattern. It can be represented as a table: for each word, a row of numbers indicating how important the remaining words are for that word. In the sentence “Anna gave Paul the book because he wanted it,” the program will focus strongly on “Paul” when processing the word “he.” That is exactly what an attention pattern is: a map of the relationships the program sees within the text.

What attention patterns reveal about a model

Modern language models like ChatGPT are barely comprehensible to humans from the outside. They contain billions of adjusted numerical values, known as parameters. No one can read these numbers directly and deduce from them why a particular answer was produced. Attention patterns are one of the few places where you can still catch a glimpse into the model.

Researchers therefore use them to understand errors. If a model cites the wrong number in a long document, one can check which part of the text it focused on while answering. Often it turns out that the attention was fixed on the wrong spot. This field of research is called interpretability — the effort to make the inner workings of AI systems understandable.

A common misconception, however, is that attention patterns show the model’s reasoning. They do not. They show which information was passed along, not why the result turned out the way it did. Anyone who derives a complete explanation from a pattern is significantly overestimating its explanatory power.

How the weights within the model arise

The mechanism behind this is called attention. The model generates three lists of numbers for each word. One functions like a search query, one like a name tag, and one like the actual content. If the search query and the name tag match well, the weight is high. This produces a ranking of the remaining words for each word.

A model does not have just one such pattern, but very many. The computation runs in parallel across multiple heads, called attention heads, and is repeated across many layers. A large model can easily produce a thousand patterns per query. Different heads specialize on their own: one tracks punctuation marks, another tracks pronoun references, a third focuses mainly on the immediately preceding word.

No one dictates this specialization. It emerges during training, the phase in which the model learns from enormous amounts of text. It is also important to note: the weights are recalculated for each query. They are not a fixed part of the model but depend on the specific text.

Attention patterns in tools and headlines

In academic papers and model presentations, attention patterns usually appear as colored grids. Dark cells represent high weights, light cells represent low ones. Freely available tools like BertViz draw such images for your own sentences, allowing you to trace the relationships yourself.

The patterns are economically significant because of their cost. The computational effort grows quadratically with text length: twice as much text roughly means four times as much work. That is why providers are working on more economical variants that compute only a portion of the connections. When a company advertises that a model now processes a context of a million words, there is almost always such a shortcut behind it.

The patterns also play a role in safety research. When investigating how a model was manipulated by cleverly crafted inputs into producing unwanted responses, attention is the first thing examined. It reveals which part of the input the model took particularly seriously.