Causal Attention
Causal Attention is a rule in language models that ensures each word, during computation, may only look at previous words, never at later ones. Without this rule, a model could copy the answer during learning instead of practicing genuine prediction.
Programs like ChatGPT write text by repeatedly guessing the next word. For this to work, they must look at the text so far while computing. Causal Attention is the rule that determines how far they may look while doing so: only backward, never forward. Every position in the text may therefore use everything to its left, but nothing that follows to its right. The word “causal” here means “related to cause”: as in the real world, only the past may influence what comes next. This is why the procedure is also called masked attention, because the future is literally covered up.
Why a model must not see the future
A language model learns from vast amounts of text. The training task is always the same: here are a few words, which one comes next? The correct answer sits right next to it in the training text. If the model were allowed to look there, the task would be worthless.
This is comparable to an exam where the solutions are printed right below the question. Anyone who copies gets full marks and learns nothing. In actual use later on, there are no more solutions, since the text doesn’t exist yet. A model without Causal Attention would shine during training and fail in operation.
There is a second, practical reason. Because each text position depends only on the previous ones, the model can practice all positions of a sentence simultaneously during training. It computes for hundreds of positions in parallel, without cheating itself. This makes training enormously faster than if one had to proceed word by word.
The mask within the attention mechanism
Modern language models work with a procedure called attention. In doing so, the model calculates for each word how strongly it relies on every other word in the text. The result can be imagined as a large table: rows are the words currently being processed, columns are the words being looked at. Each cell contains a number indicating the weight.
Causal Attention intervenes exactly in this table. All cells above the diagonal, meaning all glances into the future, are blocked. Technically, they are set to minus infinity before the weights are normalized. Afterward, their share is exactly zero. What remains is a triangular shape, in which each row reaches only to the left.
When actually generating text, this helps in yet another way. Since the part already generated never changes, the model can save its intermediate results and reuse them. This intermediate store is called the KV cache and is the reason why answers appear fluidly word by word. Without the causal rule, everything would have to be recomputed with each new word.
Text generators versus text understanders
All well-known chat and writing models use Causal Attention. This applies to the GPT series, to Llama, Claude, Gemini, and their relatives. That is why they are also called decoder models or autoregressive models. When technical texts or company announcements speak of a “decoder-only” model, this rule is always behind it.
There is, however, an important counter-world. Models like BERT, which are meant not to write text but only to understand it, deliberately forgo the mask. They may look in both directions, which helps for example with search or with sorting customer emails. For such tasks, viewing the whole sentence is an advantage, not cheating.
A common misconception is that Causal Attention has something to do with cause-and-effect knowledge. A model does not understand, through the mask, why things happen. The term refers exclusively to reading direction. And one more consequence is worth remembering: because the model cannot look ahead, it does not plan a sentence in advance. It commits word by word and cannot later take back an awkward phrasing.