Two-Tower Retrieval Model
A two-tower retrieval model is a search method in which two separate program components independently translate the search query and the searched content into lists of numbers. Because the content can already be converted in advance, the system finds the best matches from millions of entries within milliseconds.
When you search for something on a streaming service, the system has to find the right matches from millions of titles in a flash. A two-tower retrieval model solves exactly this task. It consists of two separate parts, the so-called towers. One tower turns your search query into a long list of numbers. The other tower does the same thing with every single movie, product, or text passage in the database. If two such number lists match well, the content counts as a fitting answer.
Why two separate towers are faster than one
You could also have a single program evaluate the search query and the content together. That would be more accurate, since the program could compare both texts directly with one another. But it would have to perform this calculation separately for every candidate. With ten million movies, that would mean ten million calculations per search. No user would wait that long.
The trick of the two-tower design is separation. Because the content tower doesn’t know the search query at all, it can do its work long in advance. All movies are converted into number lists once overnight and stored. When a search query then comes in, the system only has to run a single tower. The rest is a pure numerical comparison, and computers are extremely good at that.
The price for this is accuracy. The two towers never see each other, so they cannot establish fine-grained connections between the query and the content. That’s why the method is usually used only as a first stage. It sifts out a few hundred candidates from millions of entries. A more precise, slower model then re-ranks this shortlist.
How text becomes a list of numbers
Each tower is a neural network, that is, a program that has learned from examples. Its output is a vector: a list of, say, 256 numbers. You can imagine this list as coordinates in a very high-dimensional space. Content with similar meaning ends up close together there. The distance between two points thus measures the similarity in content.
This is learned using pairs that are known to belong together. One example is a search query and the title that the user actually clicked on afterward. During training, the model is adjusted so that such pairs move closer together. At the same time, random, mismatched pairs are pushed apart. After millions of such examples, both towers have learned a shared space, even though they operate completely separately.
Searching through the stored vectors is handled by a specialized database. It doesn’t stubbornly compare all entries, but instead uses a kind of pre-sorting by neighborhoods. This allows it to find the nearest points within a few milliseconds. This search isn’t 100 percent exact, but the few mismatches barely matter in practice.
From product search to AI chatbot
Recommendation systems of major platforms are almost always built on this principle. On YouTube or in online shops, the initial shortlist has to come from a huge catalog. There, one tower often contains no text at all, but rather a user’s profile: past clicks, time spent, device. The other tower describes the video or product. The principle remains identical.
In news about AI, this design usually comes up in connection with RAG, that is, with chatbots that look things up in a document collection before answering. This lookup is exactly what a two-tower model handles. It retrieves the text passages that match the question and passes them on to the language model. A common misconception is that the language model itself does the searching. In fact, a separate, much smaller retrieval model sits in front of it.