Task Compaction

Programs like ChatGPT can only look at a limited amount of text at once. This upper limit is called the context window: everything the program currently has in view must fit inside it. In a long conversation or a task with many working steps, this space eventually runs low. Task compaction is the way out: the program writes itself a summary of the history so far and continues working with only that summary. The detailed history is discarded in the process. You can picture it like transcribing messy notes into a clean record before throwing away the scratch paper.

Why long tasks fail without compaction

Modern AI assistants handle tasks that consist of dozens of steps. A coding assistant reads files, changes code, runs tests, reads error messages, and makes corrections. Each of these steps produces text that ends up in the context window. After an hour of work, the space is full, even though the task isn’t finished yet.

Without compaction, there would only be two bad options. Either the program aborts with an error message. Or it simply forgets the beginning, and with it often the actual assignment. Both are worthless to the user. Task compaction turns a hard limit into a soft one: the session keeps running, just with less detailed knowledge of the past.

Then there’s the cost. Providers bill based on the amount of text the model processes. A full history is read in its entirety with every new request and incurs costs anew each time. A summary of 2,000 words is significantly cheaper than a record of 200,000 words.

What is kept and what is discarded when summarizing

The process is usually automatic. The system monitors how full the context window is. Once the fill level crosses a threshold, say 80 percent, compaction is triggered. The model is then given the task of recording its own history: What was the task? What has already been done? What decisions were made? What is the next step?

This summary completely replaces the old history. Only a few parts remain untouched, such as the user’s original instruction and the most recent working steps. What gets discarded is mainly raw material: complete file contents, long outputs from test programs, failed attempts. Instead of the complete text of a file, all that remains is a note that it was read and what was important in it.

This is exactly where the risk lies. Whatever isn’t in the summary is permanently lost. If the model summarizes clumsily, it will afterward repeat mistakes it had already discarded before. That’s why some systems additionally store important intermediate states in files. These files survive compaction because they don’t reside in the context window but on disk.

Task compaction in coding assistants and agents

The term is most commonly encountered in AI tools for programmers. When a message like “Compacting conversation” appears in the middle of a long session, this exact process is taking place. Some tools also offer a command that lets you trigger compaction yourself before space runs low on its own.

A second area is so-called agents. These are AI programs that work through a task independently over an extended period, such as researching something on the internet. Such systems often run for hours and in doing so generate far more text than fits into a context window. Without regular compaction, they wouldn’t be technically possible at all.

A common misconception is that task compaction is a memory. That’s not the case. A memory stores information permanently and across sessions. Compaction, on the other hand, is an emergency measure against lack of space within a single session. It doesn’t make the model smarter, it just keeps it able to keep working.