Vector Store

A vector store is a database that stores information as so-called vectors — that is, as ordered lists of numbers. To do this, an AI system converts texts, images, or other content into such number lists. Two texts that are similar in meaning end up as similar number lists in storage. This sounds abstract, but it has a practical effect: the database can determine, for a new question, which stored entries are closest in meaning — not by keywords, but by meaning. That’s why a vector store is particularly well suited as a long-term memory for AI applications.

Significance for modern AI systems

Large language models — AI systems that can understand and generate text — have a fundamental problem: they can only process a limited amount of text at once. This limit is called the context window. So a model cannot keep a thousand documents in view at the same time, even if they were relevant.

A vector store elegantly solves this problem. It searches through all stored documents in advance and delivers only the most relevant passages to the model. The model then receives not the entire knowledge base, but a targeted excerpt. This combination of database search and language model is called Retrieval-Augmented Generation, or RAG for short.

Without vector stores, it would hardly be possible to connect a language model with a company’s own documents, current news, or private notes — at least not efficiently. The vector store makes knowledge outside the model searchable, without the model itself needing to be retrained.

How the search for meaning works

Before texts end up in the vector store, they are run through a so-called embedding model. This model translates language into number lists — typically with several hundred to a thousand entries per list. The trick: similar meanings produce similar number lists. “Dog” and “puppy” end up mathematically closer together than “dog” and “stock.”

When a search query comes in, it too is converted into such a number list. The vector store then compares it with all stored lists and returns the most similar ones. This comparison works mathematically via the distance between points in a multidimensional space — a concept from geometry, applied here to language.

An important difference from classic search engines: a normal database search only finds exact matches or known keywords. A vector store also finds content that expresses the same idea in different words. If you search for “What does a train ticket to Berlin cost?”, it can find an entry that only talks about “rail fares” and “capital city” — because the meaning is close enough.

Vector stores in products and in the press

In practice, vector stores show up anywhere AI works with large, changing bodies of knowledge. Corporate chatbots that need to know internal manuals use them, just as legal research tools that search thousands of court rulings do. Well-known software projects in this space include Pinecone, Weaviate, Chroma, and Qdrant — the latter is open-source, meaning anyone can use it for free.

OpenAI has also built an integrated vector store into its Assistants API, so developers can upload documents directly and make them searchable. In the tech press, the term often comes up in connection with RAG systems or the building of so-called knowledge bases — structured collections of knowledge for AI applications.

A common misconception: many people think a vector store replaces the language model or is itself an AI. In fact, it is just a database — albeit one tailored to the peculiarities of AI-generated numerical representations. The actual intelligence still resides in the model. The vector store merely ensures that the model gets the right information at the right time.