Parametric Memory
Parametric memory is the knowledge that an AI model has built into its own numerical values during training. It is locked into the model, cannot be read as text afterward, and is difficult to correct after the fact.
At its core, a language model is a huge collection of numbers. These numbers are called weights, and they determine which word the program outputs next. During training, the program reads enormous amounts of text and repeatedly adjusts its weights slightly. Whatever knowledge sticks afterward is not stored anywhere as a sentence, but only in these numbers. This knowledge built into the numbers is exactly what is called parametric memory. You cannot open it up and leaf through it, you can only retrieve it by asking questions.
Why models invent facts
A book stores knowledge word for word. Parametric memory stores it more like a memory. What remains is the approximate context, not the exact wording. That’s why a model can get the gist of something right and still cite the wrong year.
This explains a famous weakness: models sometimes give completely made-up answers in a thoroughly convincing tone. To the model, a fabricated source feels just as plausible as a real one. Both are just a pattern in its numbers. A distinction between “learned” and “guessed” is not marked in parametric memory.
A second problem is the age of the knowledge. Training ends on a certain day, and everything after that is missing. Anyone who asks a model about the current stock price gets, at best, a value from back then. Parametric memory becomes outdated without the model noticing.
How knowledge gets into the weights
During training, the model is given passages of text and has to predict the next word each time. If it’s wrong, millions of weights are shifted by tiny amounts. After many billions of such steps, the model responds to “The capital of France is” with fairly confident “Paris”. This fact isn’t stored in one place, but distributed across very many weights.
This distribution is the reason why correcting it later is hard. You can’t simply change an entry like in a spreadsheet. There is research on model editing, meaning the targeted rewriting of individual facts within the weights. In practice, however, such an intervention often breaks something else elsewhere.
As a counterpart, there is non-parametric memory. Here, the knowledge lies outside the model, for instance in a database or in documents. The model looks up matching passages of text and reads them along while answering. This method is called Retrieval Augmented Generation, or RAG for short. The advantage: you simply update the documents, not the model.
What this means for chatbots and companies
When a chatbot answers without internet access, it is working purely from parametric memory. When it displays links or quotes from an uploaded file, knowledge is being added from outside. The difference matters in practice: answers from memory are fast but poorly verifiable. Answers with a source can be checked.
That’s why, for internal assistants, companies almost always rely on external documents. A bank doesn’t want a bot that reconstructs contract terms from memory. It wants a bot that looks up the current version in the document. Parametric memory then mainly supplies language, logic, and general knowledge.
In trade news, the term also comes up in debates about copyright and data protection. If training texts continue to live on in a model’s memory, it’s unclear whether and how they can be removed again. The technical term for this is machine unlearning. A common misconception, by the way, is confusing parametric memory with chat history. The history is short-term context and disappears, while the memory in the weights remains.