Naive RAG

Programs like ChatGPT make up answers from what they saw during training. That’s why they know nothing about a company’s price list or a school’s house rules. One solution: have the program look things up in its own document collection before answering. That is exactly the idea behind Naive RAG, the simplest version of this lookup process. The user’s question triggers a search, the retrieved passages are appended to the question, and the program formulates an answer from them. “Naive” here doesn’t mean stupid, but simply: there is exactly one search pass, without checking, without follow-up questions, without a second attempt.

The standard setup that almost every project starts with

Naive RAG is the architecture that practically every company tries first. The reason is simple: you don’t need weeks of computing time to modify the model itself. You just put your own documents into a searchable store. A working prototype can be built in an afternoon this way.

The second advantage concerns traceability. Because the system uses concrete passages, it can output them alongside the answer. The user can thus see which document the answer came from. With a model that answers purely from memory, this is impossible.

It’s important to distinguish this from the more refined variants. Experts speak of “Advanced RAG” and “Modular RAG” when additional steps are added: the question is rephrased, the search results are re-ranked, the system searches multiple times. Naive RAG is the starting point of this development. Anyone who reads the term in a technical article usually finds a list of its weaknesses right next to it.

From question to answer in four steps

First, the documents are prepared, long before anyone asks a question. They are broken down into small chunks, usually a few hundred words long. Each chunk is converted into a long string of numbers that describes its meaning. Texts with similar content get similar strings of numbers. These strings of numbers end up in a special database.

When a question then comes in, it too is converted into such a string of numbers. The system searches out the chunks with the most similar strings of numbers, typically the best three to five. These pieces of text are sent to the language model together with the question. The instruction is roughly: answer the question using only this information.

You can imagine it like an exam with permitted aids. An assistant hands you a few slips of paper from a filing cabinet, selected by keyword similarity. He doesn’t read them, he doesn’t check whether the answer is actually in there. If the wrong slips arrive, you still write something down anyway. This is exactly where the weakness lies: if the results don’t fit, the model often invents a plausible-sounding answer.

Company chatbots and the point where it tips over

Naive RAG is built into a great many tools people use every day. These include search functions in company wikis, chat assistants on support pages, and programs that let you ask questions of PDF files. “Ask your documents” features in note-taking apps also usually work on this pattern.

The term shows up in business news when it comes to failed AI projects. A typical pattern: the demo with twenty documents is convincing, the rollout with fifty thousand documents disappoints. The larger the collection, the more often the search finds passages that are similar but factually wrong. Scattered information is also a problem. A question like “How has revenue developed over five years?” needs data from five reports, not from three snippets of text.

A common misconception is that RAG makes false statements impossible. That’s not true. Naive RAG reduces the number of fabrications, but it doesn’t eliminate them. That’s why providers invest a lot of money in better search methods and in verification steps after the answer is generated. Anyone who understands the term also understands what the industry is currently arguing about.