Reranking

Anyone searching for something gets a list of results. The order of this list decides almost everything, because most people only read the first entries. Reranking means that such a list is sorted a second time. First, a fast method gathers many candidates, for example a hundred documents. Then a more elaborate method examines each of these candidates more closely and reorders them. The result is the same set of hits, but in a better order.

Why the first hit list rarely has the best order

The initial search has to run over huge amounts of data. With millions of documents, you can’t carefully evaluate each one individually. That’s why this step works with shortcuts: it compares rough numerical signatures or counts matching words. This is lightning fast, but imprecise. A document can contain many matching words and still not answer the question.

An example: someone searches for “How do I cancel my phone contract early?”. The initial search finds pages containing the words phone contract, cancel, and early. Among them is also advertising for new contracts. Only a more precise check recognizes which page actually explains the notice period. Reranking ensures that this page moves up to first place.

This is especially important when a language model further processes the hits. Such models only get a limited amount of text as a basis, often just the top five documents. If the correct source is in position forty, it simply doesn’t get included. The model then answers confidently and incorrectly. A good reranker prevents exactly this error.

How a reranker reads question and text together

The difference lies in when the question and the document meet. In the fast preliminary search, each document is already converted into a numerical sequence and stored beforehand. The question is likewise converted into numbers. Then the system only searches for which numerical sequences are similar to each other. The question and the document are therefore never considered together.

A reranker does it the other way around. It receives the question and the document as a single piece of text and reads both together. In the end, it outputs a score, for example 0.92 for a very good match and 0.04 for a poor match. This construction is called a cross-encoder, because question and text are processed intertwined within the model. It recognizes subtleties such as negations or time references significantly better.

The price for this is computing time. Nothing can be precomputed, because every new question forces a new check of every document. With a million documents, this would be unaffordable. With a hundred pre-sorted candidates, it’s feasible. That’s why reranking is almost always a second step and never the first. You can think of it like a job application: a computer first roughly pre-sorts a thousand résumés, then a human conducts fifty interviews.

Reranking in search engines, online shops, and AI assistants

Large search engines have worked this way for years. Online shops also use this principle when they reorder products according to fit, ratings, and availability. Recommendation lists on streaming services likewise arise in two stages. None of this is visible; you only notice it through useful or useless results.

In AI news, the term mostly appears in connection with corporate chatbots. Such systems search internal documents and formulate an answer from them. This method is called RAG, short for retrieval-augmented generation, meaning text generation with previously retrieved sources. Providers such as Cohere, Voyage AI, or Jina sell ready-made reranker models for this purpose. They advertise that answer quality can be improved without a new language model.

A common misconception: reranking doesn’t find new hits. Whatever the initial search overlooked remains lost forever. So if you only pre-sort ten candidates, even the best reranker won’t get anything out of it. That’s why, in practice, it’s better to pass along fifty to a hundred candidates. Reranking improves the order, not the content of the list.