Parametric Memory
Parametric memory is the knowledge an AI model has stored in its own numerical values during training. It is locked inside the model, cannot simply be looked up or corrected, and is the reason language models can reproduce facts – but sometimes get them wrong too.
A language program like ChatGPT consists at its core of many billions of numbers. During learning, these numbers are slowly adjusted based on vast amounts of text until the program produces useful answers. Everything the program has absorbed about the world in the process is afterward stored exclusively in these numbers. This built-in knowledge is precisely what is called parametric memory: the numerical values are called parameters, and they are the only place where what has been learned resides. There is no list of facts inside it and no text that could be looked up. When the program says that Paris is the capital of France, it doesn’t read that anywhere – it computes the answer from its numbers.
Why built-in knowledge is so unreliable
An encyclopedia can be opened and checked. A parametric memory cannot. No one can point to the spot in a large model where a particular fact is stored. Knowledge is distributed across many numbers, and the same numbers simultaneously carry countless other pieces of information.
This leads to a well-known problem: models sometimes make things up that sound plausible but are wrong. Experts call this hallucination. The reason is that the model knows no distinction between “I learned this” and “this fits well linguistically.” It produces the same kind of answer in both cases, in the same confident tone.
A second problem is time. Parametric memory stopped on the day training ended. This point in time is called the knowledge cutoff. A model from 2024 knows nothing about an event from 2026 – and it also doesn’t know that it doesn’t know.
How facts end up in the numbers
During training, the model is given text excerpts and has to predict the next word each time. If it gets it wrong, its numbers are shifted by a tiny amount. This happens billions of times over. Facts that appear often and consistently in the training texts become strongly imprinted. Rare details, by contrast, almost completely disappear.
You can imagine this like practicing vocabulary over the years. Common words eventually stick on their own. What you’ve only heard once comes back distorted at best. Models behave exactly the same way: for famous people they are fairly confident, for unknown ones they invent biographies.
That is why in practice two types of memory are combined. Parametric memory supplies a feel for language and general world knowledge. In addition, the system searches real documents before answering and places the retrieved passages into the prompt. This lookup is called Retrieval Augmented Generation, or RAG for short, and is referred to as non-parametric memory – because it lies outside the numbers and can be swapped out at any time.
Where the difference becomes visible in everyday life
You notice it most clearly with source citations. If a chatbot answers without links, it is usually working from parametric memory. If it names specific websites or files, it has looked something up. That is precisely why search engines with AI answers now almost always offer footnotes.
In companies, the difference is a matter of money. Retraining a model with new company knowledge costs computing time and has to be repeated with every change. Placing a document search alongside it is cheaper and can be updated daily. That is why most companies first rely on RAG and only rarely on additional training.
You also encounter the term in the news in connection with copyright disputes. When publishers sue, it is often about the fact that protected texts live on within a model’s parametric memory. Removing individual pieces of content from it again is technically extremely difficult – an active field of research that goes by the name Machine Unlearning.