Causal Encoder-Decoder
A Causal Encoder-Decoder is an AI model architecture made up of two parts: one part reads the entire input, while the other generates an output from it word by word, never being allowed to look into the future. This backward-looking direction is exactly what is called causal.
Many programs that process language consist of two halves. The first half reads the complete input text and converts it into an internal numerical representation. This half is called the encoder, roughly meaning encrypter or converter. The second half, the decoder, generates the output from this – but not all at once, rather piece by piece. When generating a word, it may only look back at the words already written, never at the ones still to come. This restriction is called causal, because each word may only depend on what preceded it. A Causal Encoder-Decoder is thus a model that first reads completely and then writes step by step going forward.
Why the ban on cheating is necessary during training
During training, the model receives millions of example pairs: an English sentence and its German translation. The desired output is fully present in the training material. Without the causal restriction, the model could simply peek at the fourth word while generating the third. It would produce perfect results without having learned anything. In real-world use, this look into the future does not exist, because the output is only just being created.
That’s why a so-called mask is built in. For each position, it technically blanks out all later positions. At position three, the model literally only sees positions one and two. The advantage: all positions of a sentence can still be computed simultaneously, rather than word by word in sequence. This makes training enormously faster, without any cheating taking place.
An important distinction: this ban explicitly does not apply to the encoder. It is allowed to view the input text completely in both directions. In translation, this makes sense, since the source sentence is fully available from the start. Only the output side must respect the temporal order.
Two towers and the bridge between them
You can picture this setup as an interpreter at a conference. He first listens to the entire speech and memorizes its meaning. Afterwards, he speaks the translation sentence by sentence. He cannot take back words already spoken, and he doesn’t yet know word-for-word what he is about to say. A Causal Encoder-Decoder behaves in exactly this way.
Technically, a mechanism called cross-attention connects the two halves. Attention means paying attention: the model learns to focus specifically on certain parts of the input at each output step. When translating the German verb, it thus pays particular attention to the English verb. Within the decoder, there is additionally a masked self-attention that only looks backward. Together, this results in three types of attention: unrestricted within the encoder, backward-looking within the decoder, and acting as a bridge between the two.
The process of generation always follows the same pattern. The model computes a probability over the entire vocabulary for the next word. One of these words is selected and appended to the text so far. Then the step starts over, now with one more word of context. This repeats until a special end token appears.
Translators, subtitles, and the competition from pure decoders
This architecture is found in many tools used every day. Machine translation is the classic case, as is automatic summarization of long texts. Speech recognition often works this way too: the encoder processes the audio recording, the decoder writes the text. Well-known systems such as T5 or Whisper follow this pattern.
In the news, the term usually comes up as a distinction. The large chatbots, namely, are not encoder-decoders but pure decoder models. They only know a single text stream in which the question and answer appear one after another. They are nonetheless causal, since they too write strictly from left to right.
A common misconception is that causal has something to do with investigating causes. The model understands no causes and no relationships in the world. What is meant is exclusively the direction of information flow through time. Anyone who separates these ideas will find it much easier to understand technical texts and product announcements.