Continual Learning

Programs that are today referred to as Artificial Intelligence learn from huge amounts of example texts and images. This learning phase is called training. It runs once, often takes weeks, and ends on a fixed cutoff date. After that, the program is finished and no longer changes on its own. Continual learning would be the opposite: a system that keeps learning further during ongoing operation, the way a human absorbs new things every day. So far this only works to a limited extent, and the reason is a technical problem with an impressive name.

Why models with a cutoff date age

Every large language model has a knowledge cutoff date. It only knows what was in its training data up to that point. If you ask it about an event from last week, it either has no idea or invents a plausible-sounding answer. The longer a model is in use, the more outdated its picture of the world becomes.

Today’s solution is expensive: every few months a new model is trained from scratch. Depending on its size, this costs tens of millions and consumes enormous amounts of electricity. A system that simply keeps learning instead of being rebuilt would drastically reduce this effort. That is why continual learning is considered one of the most important open research goals.

But it’s not just about costs. In some areas the world changes faster than one can retrain a model. A bank’s fraud detection system must recognize new fraud schemes immediately. A robot in a factory encounters workpieces that did not exist during its training.

The problem of catastrophic forgetting

A neural network does not store its knowledge in individual drawers. It consists of billions of numbers, called parameters. Each piece of information is distributed across very many of these numbers. When the network learns something new, the numbers are adjusted — and in the process they often fall out of the arrangement that carried the old knowledge.

Experts call this effect catastrophic forgetting. If you train a model to recognize cats and then train it exclusively on cars afterward, it may recognize cats worse than before. The knowledge doesn’t disappear neatly bit by bit, but often collapses abruptly. A human who learns Spanish does not forget their German in the process — this is precisely where our brain has so far been superior to the computer.

There are several countermeasures. In one approach, the parameters that are especially important for old knowledge are marked and only changed cautiously during further learning. In another, old examples are mixed in with the new training data so that the model keeps refreshing its prior knowledge. A third option is to attach additional building blocks to the model for new tasks while leaving the old part untouched. None of these methods fully solves the problem.

What chatbots do instead today

When a chatbot tells you the latest news, it usually doesn’t learn anything in the process. Instead, it searches the internet or a database and writes what it finds into the request. The model itself remains unchanged. This procedure is called Retrieval Augmented Generation, and it is a lookup, not learning. The memory of many assistants works the same way: your preferences end up in a notes file, not in the parameters.

A common misconception is therefore the assumption that a chatbot becomes smarter through conversations with you. That is usually not true. Your inputs may later be incorporated into a new training run, but that is a separate, planned process on the provider’s side.

In the news, the term usually appears in two contexts. First, in announcements of models with allegedly learning-capable long-term memory. Second, in robotics and autonomous driving, where systems are supposed to adapt to new environments. Anyone who wants to properly interpret such reports should always check: Does the model actually change, or is it just looking things up more cleverly?