Compaction

Compaction is the English word for condensation. What is meant is a cleanup process: a program takes a lot of accumulated data and replaces it with a shorter version that says the same thing. Anything that is duplicated, outdated, or unimportant gets thrown out in the process. You can picture it like a notebook of lecture notes. After ten class sessions, you write a two-page summary and put the loose sheets away. The term originally comes from storage technology, but today it is used just as much for AI chat programs.

Why systems eventually choke on their own baggage

Every system that continuously writes new data has a growth problem. Storage space is limited, and searching through large amounts of data becomes slower with every entry. Without compaction, a service becomes more expensive and sluggish over time, even though it isn’t doing anything new.

With language AIs, there’s an additional hard limit: the context window. This is the term for the amount of text a model can take in at once. Depending on the model, this limit ranges from a few thousand to several hundred thousand text chunks. Once it’s reached, the model can’t simply continue. Either the conversation breaks off, or the beginning is dropped without replacement.

It’s also about money. With most AI providers, you pay per text chunk processed. If the entire conversation so far is sent along with every new question, costs rise with every round. Good compaction significantly lowers this bill without the user noticing anything.

From a mountain of data to a short version

In databases, compaction usually works like this: changes are initially only appended at the end, never overwritten in the middle of existing data. This is fast, but it produces many outdated versions of the same entry. A background process later goes through all the files and keeps only the most current state per entry. Deleted records disappear for good during this pass.

With AI assistants, the language model itself takes on this work. As the conversation approaches the limit, the older part is sent to the model with instructions to summarize it. The result replaces the old history. Important decisions are preserved, such as which programming language is being used or what the user’s name is. The exact wording of old messages is gone afterward.

This is exactly where the catch lies. Compaction is always a decision about what is unimportant, and that decision can be wrong. A detail that only matters twenty messages later is no longer retrievable after compaction. Many users experience this as the assistant suddenly becoming forgetful in the middle of a conversation. Compaction should therefore not be confused with compression: compression can be reversed, compaction discards information permanently.

Where the term appears in products and reports

Compaction is most visible in AI coding tools. Assistants like Claude Code or Cursor show a notice when they condense the history so far. Some also offer the process as a command that you can trigger yourself. Those who work with it a lot quickly learn to repeat important instructions once more after a compaction.

The same technology runs unnoticed in the background of many online services. Databases like Cassandra, RocksDB, or LevelDB compact their files regularly, and messaging systems like Kafka clean up old message streams this way. In technical post-incident reports, compaction often appears as a cause: the cleanup process briefly consumes so much computing power that the actual requests slow down.

For investors and industry observers, the term is interesting for another reason. It shows that large context windows alone don’t solve the memory problem. As long as models can only take in a limited amount of text at once, compaction remains a central lever for the cost and quality of AI products.