Compaction Summary

Programs like ChatGPT can only look at a limited amount of text at once. This upper limit is called the context window: everything the program currently has in view for generating a response. In a long conversation, this limit is eventually reached. At that point, the previous history is replaced by a short summary, and the conversation continues using this condensed version. This automatically generated condensed version is exactly what is called a compaction summary. The English term “compaction” means something like condensing or compressing.

Why long conversations would otherwise break off

Without compaction, an assistant would have two bad options. Either it refuses to answer because the history no longer fits, or it simply discards the oldest messages. Both are unpleasant for the user, since important agreements made an hour ago would suddenly be gone.

There is also an economic reason. Providers usually bill per processed text unit. If the entire history is sent along with every new question, costs rise with each round. A 500-word summary is significantly cheaper than a 50,000-word transcript. Response time also decreases, since less text needs to be read.

This is especially important for coding assistants. Such sessions often run for hours and involve hundreds of work steps. Without compaction, extended collaboration on a software project would hardly be technically possible.

What is kept and what is discarded during compaction

The process is usually the same. The system monitors how full the context window is. If a threshold is exceeded, for example 80 percent, the older part of the conversation is sent to the language model itself. The instruction is roughly: summarize what is needed for further work. The result takes the place of the original, while the most recent messages are often left unchanged.

What matters is what counts as important. Good instructions explicitly require certain content: the user’s goal, decisions made, files worked on, failed attempts, and open tasks. Pleasantries and side questions are dropped. You can think of it like meeting minutes: not every word, but every resolution.

The catch lies in the nature of the process itself. A summary is a loss of information, and it is generated by the very same fallible AI that then continues working with it. If a detail is missing, it is gone for good. If compaction happens repeatedly in succession, you end up with a summary of a summary, and inaccuracies compound. A related but different approach is external memory: notes are written to files and selectively reloaded when needed, instead of being compressed.

The hint in the middle of the chat

As a user, you notice compaction through small hints. Some interfaces display a line like “Conversation has been summarized” or a divider in the history. In developer tools, there is sometimes a command that lets you trigger compaction yourself before memory runs low.

Many people are familiar with a typical symptom without knowing the term. After a long time, the assistant suddenly seems forgetful and asks about something that was already settled long ago. Often, compaction is behind this, with that particular detail not having made it into the summary. A brief reminder of the earlier agreement usually helps in that case.

In trade press, the term mainly comes up in connection with so-called AI agents, i.e., programs that independently work through tasks over extended periods. Providers describe in their documentation how their system performs compaction and at what utilization level. For companies, this is not a minor detail, since it determines how reliably an assistant works over the course of hours.