Chroma

Chroma is a store for queries that work by meaning rather than by exact wording. To do this, every text is first converted into a long list of numbers. This list of numbers describes the content in such a way that similar content gets similar numbers. Chroma stores these lists and, on request, finds the entries whose numbers are closest. This way, a search for “How do I cancel?” also finds a paragraph that only mentions “contract termination.” The software is free, and its source code is publicly viewable.

Why chatbots need a store like this

A language model like ChatGPT only knows what appeared in its training. It does not know your class assignment, your company’s manuals, or last week’s price list. But you can hand it these texts before it answers. That is exactly what you need a store for—one that quickly retrieves the matching excerpts.

This approach is called Retrieval Augmented Generation, or RAG for short: search first, then have it answer. Chroma is one of the best-known tools for the search part. It has become widespread because it can be started with just a few lines of code. Other systems of the same kind are Pinecone, Weaviate, or Qdrant.

This matters economically because many companies do not want to train their own models. Training costs millions. A document store alongside a ready-made model costs almost nothing. That is why a large part of corporate AI today follows this pattern.

From document to list of numbers

First, a document is cut into pieces, usually a few paragraphs long. Each piece is passed through a small helper model, a so-called embedding model. It returns a list of often several hundred numbers. You can imagine this list as coordinates in a space. Texts about football end up close together there, texts about tax law somewhere else entirely.

Chroma stores these coordinates together with the original text and additional information such as file name or date. When a question comes in, it too is converted into coordinates. Chroma then searches for the nearest points. With many entries, it does not check all of them but uses a shortcut that almost always finds the right hits.

A common misconception: Chroma does not understand the texts and does not formulate answers. It is purely sorting and search technology. The language model writes the answer afterward. And if the pieces are cut poorly, even the best search only finds fragments.

Where Chroma is used

Chroma is most often found in chatbots that answer questions about a specific collection of documents. Examples include an assistant for a machine manufacturer’s user manuals or a search tool for a newspaper’s archive. Coding assistants also use the technology to find matching code sections in large projects.

In news reports, Chroma usually appears as part of the so-called AI tool chain. The name comes up alongside frameworks like LangChain or LlamaIndex, which assemble the process of searching and answering. The company behind Chroma has raised venture capital in the tens of millions and earns money through a cloud-hosted variant.

For learners, Chroma is interesting because you can try it out on your own laptop. A few PDF files, a script, and the search works. However, with very large amounts of data and many simultaneous users, companies often turn to specialized alternatives.