Retrieval Augmentation
Retrieval augmentation means that an AI language program looks up matching passages in a document collection before answering and incorporates these findings into its response. This allows it to also answer questions about current or company-internal content that it never encountered during training.
Programs like ChatGPT generate answers from what they absorbed during training from vast amounts of text. This knowledge is fixed within the program and ends on the day training was completed. Retrieval augmentation breaks through this limit. Before each answer, the system first searches a connected collection of documents for matching passages. These found passages are handed over to the program together with the question. It no longer answers purely from memory, but with an open book in front of it.
What helps against fabricated answers
Language programs tend to make up things that sound convincing. Experts call this hallucination. The reason is simple: the program predicts word by word what is likely to come next. It does not check whether a statement is actually true. If the correct passages are laid directly in front of it, the rate of fabrication drops significantly.
On top of that comes a very practical advantage for companies. A corporation cannot train its entire internal knowledge into a model. A new training run costs millions and takes weeks. A document collection, on the other hand, can be updated every day. If a contract is changed, the system knows the new version immediately.
A third point is traceability. Because the system knows which documents it drew from, it can cite sources. A user can look up whether the answer really appears in the named file. With a pure language model without retrieval, this is impossible, because there the knowledge is smeared across billions of numerical values.
The path from question to matching passage
First, all documents are broken down into small chunks of a few paragraphs each. Each chunk is converted into a long sequence of numbers, a so-called vector. This sequence of numbers describes the meaning of the text. Two texts with similar content receive similar sequences of numbers, even if they use completely different words. All vectors end up in a special database.
When someone asks a question, that question is also converted into a vector. The database then searches out the text chunks with the most similar sequences of numbers, typically three to ten pieces. Unlike a classic keyword search, meaning counts, not exact spelling. Someone asking about 'vacation days' will also find a paragraph about 'leave entitlement'.
In the final step, the system builds an expanded query. It contains the found text chunks, the original question, and an instruction such as: answer only based on these excerpts. Only this package goes to the language model. The quality depends almost entirely on the search step. If the database finds the wrong passages, even the best language model no longer helps.
From company chatbot to web search
Most chatbots on company websites work according to this principle today. A car manufacturer’s manual assistant searches through the owner’s manual, a bank’s support bot searches its forms. Search engines with AI answers also use retrieval augmentation: they fetch web pages and summarize them. The source links beneath such answers are a direct telltale sign.
In tech industry news, the term usually appears under the English abbreviation RAG, for Retrieval Augmented Generation. An entire industry has grown up around this method, such as providers of vector databases like Pinecone or Weaviate.
A common misconception is that retrieval augmentation makes the model smarter. It does not. It merely supplies better ingredients. A weak model still draws weak conclusions from good passages. The method should also be distinguished from fine-tuning: there, the model itself is altered, whereas here it is left untouched and only the query is supplemented.
