Knowledge Base

A knowledge base is an organized collection of information on a specific subject area. Unlike a loose folder full of files, it is structured so that one can search it in a targeted way. Typical contents include manuals, product descriptions, legal texts, incident reports, or answers to frequently asked questions. The purpose is always the same: knowledge that would otherwise exist only in people’s heads or scattered documents should be retrievable in one place. The term dates from before today’s AI systems, but has taken on new meaning through them. That’s because a computer program can search such a collection without having memorized it beforehand.

Why language models need a reference work

A language model like ChatGPT absorbed its knowledge during training. It has seen huge amounts of text and derived patterns from them. This knowledge is then stored as numbers within the model, not as lookupable sentences. This has two drawbacks. First, it ends at a cutoff date: the model doesn’t know what happened after that. Second, the model cannot say where a statement came from.

On top of that, there’s a well-known problem: language models sometimes invent facts that sound convincing but are wrong. Experts call this hallucination. A knowledge base, by contrast, is an effective remedy. The model is presented with the relevant passages of text and must base its answer on them. If someone asks about an online shop’s return policy, the system retrieves the actual section from the terms and conditions.

For companies, there’s another important factor: internal knowledge often must not leave the building. Retraining a model on it would be expensive and would have to be repeated with every change. A knowledge base, on the other hand, can be updated within minutes. You change a document, and the next answer is already up to date.

From file to searchable entry

The first step is splitting things up. Long documents are cut into sections of a few hundred words, so-called chunks. This is necessary because a model can only process a limited amount of text per query. A 300-page manual doesn’t fit, but the one relevant paragraph does.

Afterward, each section is translated into a long string of numbers, a so-called embedding. These numbers describe the meaning of the text. Sections with similar content get similar number sequences. When someone asks a question, it too is translated into numbers. The system then searches for the sections with the most similar values. This way it also finds matches in which not a single word of the question appears literally.

These retrieved sections are sent to the language model together with the question. It formulates an answer from them and ideally names the source. The entire process is called Retrieval Augmented Generation, or RAG for short. It’s important to distinguish: the knowledge base is the stockpile, RAG is the procedure that searches within it. A common misconception is that the model learns something new in the process. That’s not true — it only reads the text for this one answer and forgets it again afterward.

Support chats, corporate assistants, and search engines

Knowledge bases are most commonly encountered in customer service. A mobile carrier’s chatbot draws on rate overviews and instructions. The help sections of many websites are likewise nothing other than a knowledge base for humans. Within companies, internal assistants run that search operating manuals, project reports, or HR policies.

In the news, the term usually comes up in connection with enterprise software. Vendors advertise that their AI “works on your own data.” This almost always refers to exactly this setup. AI search engines like Perplexity also work on this principle, except their knowledge base is the open internet.

One warning is in order: a knowledge base is only as good as its content. If it contains an outdated price list, the system will answer with outdated prices — phrased very convincingly. Maintenance is therefore not a side issue, but the actual work.