OpenSearch

OpenSearch is a program that companies install on their own servers to make their own data holdings searchable. You can think of it as a search engine you set up yourself. It does not search the internet, but whatever you feed into it beforehand: product descriptions from an online shop, articles from a news archive, or the log files of thousands of computers. The program’s source code is openly accessible; anyone may read it, modify it, and use it free of charge. The project was launched in 2021 by the cloud provider Amazon Web Services, and since 2024 it has belonged to a neutral foundation, the Linux Foundation. The package also includes an interface called OpenSearch Dashboards, which displays search results as charts and tables.

The fork from Elasticsearch

OpenSearch is not a new development but a copy with its own future. The original is called Elasticsearch and was for years the standard software for search and log analysis. In 2021, its maker Elastic changed the license, that is, the legal terms of use. After that, providers were no longer allowed to simply resell Elasticsearch as a paid cloud service. Amazon did exactly that and suddenly found itself without a legal basis.

The answer was a so-called fork: you take the last freely usable version of the source code and continue developing it under a new name. Such forks are an important source of power in the world of open software. They show that a single company cannot restrict a project at will, as long as the code remains freely available.

For companies, predictability is what matters most. Anyone who bases their entire log analysis on a piece of software doesn’t want to be surprised by license fees two years later. OpenSearch is licensed under Apache 2.0, which imposes hardly any restrictions. That is precisely its biggest selling point compared to the original.

How the index makes search fast

The core of OpenSearch is an inverted index. A normal book lists chapters and, within them, words. An inverted index does it the other way around: it lists words and, alongside them, the documents in which they appear. Just like the index at the back of a schoolbook. When you search for a word, the program doesn’t have to read all the millions of documents, just look up a single entry.

For this to work even with huge amounts of data, the index is split up. The parts are called shards and reside on multiple machines at once. A search query goes out to all the parts, each returns its hits, and the result is assembled from them. Copies of each part also exist. If a machine fails, a copy takes over, and the search keeps running.

Newer versions can also search by meaning instead of by exact word match. To do this, an AI model converts every text into a long sequence of numbers, a vector. Texts with similar meaning receive similar number sequences. A search for “repair a bicycle” then also finds a text about “fixing a flat tire”. This vector search is the reason OpenSearch is so often mentioned in connection with AI applications.

Search fields, server monitoring, and AI chatbots

Most often you encounter OpenSearch without noticing: as the search field of an online shop or a newspaper archive. The suggestions that appear while typing often come from such software too. The second major area of use is monitoring technical systems. Servers constantly write logs, and when a website goes down, the team searches through them for the cause. With millions of lines per hour, that’s not possible without a search index.

In the news, the name comes up in two contexts. First, in disputes over open-software licenses, where OpenSearch serves as a textbook example. Second, in the AI field: chatbots meant to answer questions based on company documents need a database that retrieves matching text passages. This method is called Retrieval Augmented Generation, and OpenSearch is a common choice for it.

A common mix-up is confusing it with the OpenSearch standard, an old format for search fields in browsers. The name is the same, but the thing itself is different. A distinction should also be made between the free software and the Amazon OpenSearch Service, the paid cloud offering in which Amazon handles operation and maintenance.