Fan-out query

When you type something into a search engine, it looks like a single question. In the background, though, it often turns into a whole bundle of questions. The system breaks your sentence down into several sub-questions and sends them off simultaneously to different places. It then collects all the answers again and builds a result from them. This exact process of fanning out and merging back together is called a fan-out query. The name comes from the English phrase “to fan out,” meaning to spread out like a hand fan.

Why a question is rarely just one question

Many real questions consist of several hidden sub-questions. Take “Which e-bike under 2000 euros has the best range and fits in my trunk?”. Hidden within it are at least three conditions: price, range, and size. A system that only searches for the complete sentence will probably find nothing suitable. A system that checks the conditions individually has significantly better chances.

Then there’s speed. The sub-queries run in parallel, meaning simultaneously rather than one after another. Ten queries that each take half a second individually still only cost about half a second in total when combined. For users, this feels like a single fast search. Without this principle, AI answers with cited sources would hardly be usable in their current form.

For businesses, this has an unpleasant side effect. In the past, you could optimize a website for a specific search term. But if the search engine turns that into ten of its own sub-queries, nobody knows exactly what to optimize for anymore. A large part of the current discussion about the future of search engine optimization revolves around exactly this problem.

From fanning out to putting it back together

The first step is decomposition. A language model, meaning an AI system for text, reads the original question and formulates several variants from it. Some of these are rephrasings, others cover individual aspects, and still others add technical terms the user wasn’t aware of. In this way, a single input quickly turns into five to thirty search queries.

In the second step, these queries are sent off, often to quite different destinations. One goes to the web index, one to a product database, one to map data, one to current news. Each source returns its own list of results. These lists are initially unsorted and contain many duplicates.

The third step is merging, known in technical jargon as “fan-in.” Duplicate hits are removed, the results are re-sorted by relevance, and a language model formulates an answer from them. This is also where the biggest weak point lies. If a sub-query was poorly formulated or a source responds slowly, that piece is missing from the final answer. The user doesn’t notice this and considers the result complete.

Where fanning out happens everywhere

This principle is most visible in AI search services. Perplexity, ChatGPT with web search, and Google's AI Mode often display how many search queries they’re currently making. If it says “searched 12 sources” there, you’ve just seen a fan-out query in action. With so-called deep research features, which work for minutes at a time, there can be hundreds of sub-queries.

The pattern is old outside of AI as well. A travel comparison site queries dozens of airlines simultaneously with every search. Your social media homepage is created by the service collecting posts, ads, and notifications in parallel. In software development, fan-out has therefore been a standard term for distributed systems for years.

A common misconception is that more sub-queries automatically mean better answers. Every additional query costs computing time and money, and poor sources can actually make a good answer worse. Providers are therefore constantly searching for the right balance between thoroughness and effort.