BrowseComp

BrowseComp is a collection of 1,266 very tricky search questions. It was published in 2025 by the company OpenAI, which also develops ChatGPT. It is used to test programs that are allowed to search the internet on their own. Each question has exactly one correct answer, usually just a name, a date, or a number. This makes it possible to count unambiguously how many answers are correct. The name is an abbreviation for “Browsing Competition,” roughly meaning a competition in web surfing.

Why short answers are so hard to find

Such collections of test tasks are called benchmarks. For the AI industry, they are what report card grades are for school: a yardstick for comparison. Many older benchmarks consist of knowledge questions that a program can answer from memory. BrowseComp works differently. The questions are deliberately phrased so that no model knows the answer by heart.

A typical example sounds like this: The task is to find an athlete who took part in a certain tournament between 1990 and 1994, was born in a country with a certain characteristic, and later changed clubs. Each individual condition fits many people. Only all of them together yield one person. To find them, a system has to combine several sources.

That is exactly what makes the test interesting for companies. They want to know whether an AI can research independently, rather than just chat. Success rates were low at the start: simple chatbots without internet access scored below two percent. Even with a search function, many systems stayed below ten percent. Specialized research programs later achieved significantly higher scores.

How the questions were constructed

The tasks were built backwards. People first picked an answer, for example a particular music album. Then they wrote a question that fits exactly that album without naming it. Afterwards, they checked two things. First: does a search engine fail to find the answer among the first results? Second: do the best AI models of the time fail at it?

Only questions that passed both tests made it into the collection. As a result, BrowseComp contains almost exclusively tasks where mere lookup is not enough. In addition, it was measured how long humans need to solve them. For many questions, the test subjects gave up after two hours.

The evaluation itself is straightforward. Because every answer is short, it can be compared with the stored solution. This is a big advantage over tests where an essay has to be graded. There, experts often disagree about what counts as correct. One well-known criticism remains, though: BrowseComp only measures the finding of facts, not summarizing or weighing them.

BrowseComp in product announcements and news

The term mainly appears when companies present new research features. Such features often run under names like “Deep Research” and work on a query for several minutes. Press releases then cite a BrowseComp score as evidence of quality. OpenAI, Google, and Chinese providers like Alibaba use the test in their publications.

For investors, this is relevant because a market stands behind these numbers. Automated research affects professions such as analysis, journalism, and legal review. Whoever performs well there advertises a measurable edge. However, such figures should be read with caution. A high score on a test does not mean the system works reliably in everyday use.

There are now also offshoots. BrowseComp-ZH poses the same kind of questions in Chinese and on the Chinese web. Other variants test research in images or in scientific texts. If you see a report with a percentage figure, it is worth checking which variant is meant. The scores are not comparable with one another.