Knowledge Decay

Programs like ChatGPT learn from huge amounts of text that were collected beforehand. This collection has a cutoff date. Anything that happened after that, the program simply doesn’t know. And even more importantly: a lot of what it learned no longer holds true. Prices change, governments change, companies get renamed, records get broken. This gradual invalidation of learned knowledge is exactly what’s called Knowledge Decay. A fitting comparison is a printed encyclopedia from 2019: it isn’t written incorrectly, it’s just outdated in many places.

Why outdated answers can get expensive

The trouble starts because outdated answers don’t sound outdated. A language model states a piece of information from 2023 just as confidently as one from yesterday. There’s no warning label, no expiration date attached to the sentence. If you don’t already know the correct answer, you won’t notice the difference.

In some areas this is merely annoying, in others it’s risky. With tax rates, interest rates, medication dosages, or laws, being current is what decides right from wrong. A company that hands its customer service over to an AI system is liable for its answers. A model that still states the old return policy deadline causes real costs.

For AI model providers, Knowledge Decay is therefore also a business problem. A model loses value the longer it stays unchanged in deployment. This explains why major providers release new versions every few months. Some of these updates bring no new capabilities at all — just fresher knowledge.

Where the model’s expiration date comes from

During training, the model is shown billions of text snippets. It doesn’t store articles or tables in the process, but instead adjusts millions of internal dials. These dials are called parameters. After that, the model is fixed and no longer changes on its own. The last point in time covered by the training data is called the Knowledge Cutoff.

Decay begins at this date, and it proceeds unevenly. Mathematics, grammar, and physics barely age at all. People in office, product versions, exchange rates, and prices age within months. That’s why the same model can be completely reliable in one subject and embarrassingly wrong in the next.

There are two kinds of remedies. With retraining, the model receives new data and updates its parameters — this is effective but laborious and expensive. The usual approach instead is to feed the model current sources at runtime: a web search, a database, a company document. This method is called Retrieval Augmented Generation. The model itself stays old, but has fresh material right in front of it.

Spotting Knowledge Decay in everyday use

The problem shows up most clearly with questions about the current state of things. Ask a language model without internet access about the latest iPhone or the current head of government. Often you’ll get an answer that was correct two years ago. Some systems openly state their knowledge cutoff, others stay silent about it.

In product descriptions and technical articles, the term often appears alongside details about the training cutoff. When a provider writes that their model has knowledge up to October 2024, this is exactly the issue being addressed. Likewise, a note that a chatbot can search the web is a direct response to Knowledge Decay.

A common misconception is that Knowledge Decay is the same as hallucinating. When hallucinating, a model invents something that never existed. With knowledge decay, it correctly reproduces a piece of information that used to be true. In practice, this means: for anything that changes quickly, it’s worth taking a quick look at an up-to-date source.