Context Window
The context window is the amount of text an AI language model can keep in view at the same time. Everything within it flows into the response, everything outside it simply doesn't exist for the model.
Programs like ChatGPT always work with a limited excerpt of text. This excerpt is called the context window. It contains everything the program can currently take into account: your question, the conversation history so far, uploaded documents, and its own answer as it’s being generated. Whatever fits into this window is factored into the response. Whatever doesn’t fit simply doesn’t exist for the program. You can picture it like a desk: only what’s lying on it can be used. Everything else sits in a locked cabinet.
Why chatbots suddenly become forgetful
The size of the window determines which tasks a model can even solve. With a short window, a newspaper article might fit, but not an entire book. Anyone wanting to ask a model to summarize a 300-page contract needs a very large window. Otherwise the text has to be broken into pieces, and connections between page 12 and page 280 get lost.
You notice the limit in everyday use too. In a long chat, the first messages eventually slide out of the window. The model then seems forgetful and contradicts itself. This isn’t a memory error in the human sense, though. The old messages are simply no longer being fed in.
A common misconception: the context window is not what the model learned during training. Learned knowledge is fixed within the model. The window is merely the working memory for the current case. After the conversation it’s empty, and none of it is permanently stored.
Measured in tokens, limited by computing effort
The size is not given in words but in tokens. A token is a text building block, usually a short word or part of a word. As a rule of thumb: 1,000 tokens correspond to roughly 750 words of English text. A window of 128,000 tokens thus holds roughly a thick paperback.
Why is the window limited at all? Because computing effort rises sharply with length. In principle, the model compares every text building block with every other one in order to recognize relationships. When the text length doubles, this effort quadruples in the classic architecture. That’s why very long inputs cost more money and more time.
Research and engineering have eased the problem but not solved it. Newer methods compute more efficiently and allow windows of a million tokens or more. Still, it holds true that large windows don’t automatically mean good results. Models overlook information more often when it’s located in the middle of a very long text. Experts call this phenomenon “lost in the middle.”
A selling point in every model announcement
When a company introduces a new model, the window size is almost always in the headlines. Figures like “200K context” or “1 million tokens” have become a competitive argument. This is interesting for investors and corporate customers, since it determines whether legal files, program code, or research reports can be processed in one piece.
The window also plays a role in billing. Anyone using AI via a programming interface pays per token, meaning by amount of text. A long attachment in every request noticeably drives up costs. Many applications therefore search beforehand for the relevant passages of text and send only those into the window.
In private use, you’re most likely to run into the limit when uploading files or having very long conversations. A practical tip: it’s better to repeat important instructions anew rather than relying on earlier messages. That way they’re sure to land in the window again.