Middleware

In a larger computer system, rarely do just one or two programs run. Usually there are many, and they need to constantly exchange data. Middleware is software that sits exactly in between and organizes this exchange. The name says it literally: “middle” means the center, so it lies in the middle between other programs. An example: A phone app wants to display account balances that are stored on an old bank computer. The app and the bank computer speak completely different technical languages. The middleware accepts the app’s request, translates it, fetches the data, and sends it back in a format the app understands.

Why nobody wants to build everything themselves

Without such an intermediate layer, every program would have to know every other program directly. With five programs, that would already be ten possible connections. With fifty programs, it’s over a thousand. Someone would have to program and later maintain each of these connections. Middleware bundles this: each program only talks to the middle, not to all the others individually.

In addition, certain tasks are the same in almost every program. Checking whether a user is logged in. Logging who did what and when. Throttling requests when too many come in at once. Writing such things over and over again would be a waste. Instead, they are placed once in the intermediate layer, and all programs benefit from it.

For companies, this is also a cost issue. Old systems, often in use for decades, cannot simply be replaced. A middleware places itself in front of them and makes them usable for new applications. This is significantly cheaper than rewriting the old system.

What happens in the intermediate layer

Technically, middleware itself is just a program; often several of them run in parallel on servers. When a request comes in, it passes through a chain of checkpoints. The first might check the password. The second writes an entry into the log. The third converts the data into the appropriate format. Only after that does the request go to the actual target program, and the response runs back through the same chain in reverse.

A widespread special case is message middleware, also called a message broker. It works like a mailbox. One program places a message into it and immediately continues working. Another program picks it up later. As a result, both don’t have to run at the same time. If the recipient is briefly unavailable, the messages remain in the mailbox instead of getting lost.

A common misconception is that middleware is a specific product. It is not. The term describes a role in the system, not a fixed technology. A web framework, a database driver, and a large enterprise server can all be middleware, even though they have technically nothing in common.

Middleware in AI products and in the news

In AI applications, the intermediate layer has meanwhile become very important. Anyone integrating a chatbot into a company rarely sends the user’s question directly to the language model. Beforehand, a middleware checks whether the user is authorized, filters out sensitive content, and searches for relevant company documents. Afterwards, it tallies how much computing power was consumed so that billing is accurate.

Another case are services that connect multiple AI providers simultaneously. They accept a request and route it to different models depending on price or load. For the app behind it, nothing changes; it always talks to the same interface. Such services are a growing market and therefore regularly appear in business news.

Even outside AI, one often encounters the term without noticing it. When a travel website compares prices from twenty airlines, middleware is working in between. When a payment app talks to a store’s checkout system, the same applies. You never see it, but without it, hardly any modern system would function.