Microservices

Large programs like an online shop consist of many tasks: login, search, shopping cart, payment, shipping. You could write all of this as a single large program that runs as one piece. With microservices, you do the opposite. Each task gets its own small program that runs on its own and can be started independently. These small programs talk to each other over the network, similar to colleagues in different offices calling each other. The alternative — one large program cast in a single piece — is what developers call a monolith.

Why big companies break up their software

The most important reason is division of labor. With a monolith, all developers work on the same codebase. Once hundreds of people are writing to it, they get in each other’s way. With microservices, each team gets its own small program and can release it independently. One team can update its version ten times a day without asking anyone else.

The second reason is scaling, meaning the ability to grow along with high load. On Black Friday, it’s mainly the payment function that gets overwhelmed, not the page with the terms of use. In a monolith, you have to duplicate the entire program even though only a small part of it is overloaded. With microservices, you simply start twenty copies of the payment service and leave the rest unchanged.

Third, a failure often stays contained to one spot. If the service for product reviews crashes, the shop can keep selling and simply hides the reviews. However, this only works if you deliberately build it that way. Without this precaution, a stuck service drags the others down with it, because they wait forever for its response.

How the small services talk to each other

Each microservice runs in its own process, usually inside a container. A container is a self-contained package that holds the program along with everything it needs and starts the same way everywhere. The services exchange messages over the network, often as simple requests via the web protocol HTTP. In addition, many systems use a queue: one service drops a message there, and another picks it up later.

An important rule is that each service owns its own data. The payment service doesn’t reach directly into the shopping cart’s database. Instead, it politely asks the shopping cart service. Only this way do the parts remain truly independently interchangeable. If you violate this rule, you end up with a monolith with a network in between — the worst of both worlds.

The price for this is operational complexity. Instead of monitoring one program, you suddenly have to monitor fifty. You need tools that can trace a single order across all the services involved. And the network is less reliable than a call within a single program: requests get lost or take too long. That’s why the rule of thumb today is that small teams are better off starting with a well-organized monolith.

Microservices at Netflix, Amazon, and in AI services

This architectural style became well known through Amazon and Netflix, which broke their systems down into hundreds of services during the 2000s and 2010s. Anyone who opens Netflix today triggers requests to dozens of microservices: login, recommendations, subtitles, billing. Uber, Spotify, and most large banks operate this way too.

The principle also shows up in the AI world. A chatbot provider runs separate services for user management, document search, and the actual answer computation by the language model. The expensive model service runs on specialized graphics chips, while the rest runs on regular servers. This separation saves money, because you only need to duplicate the expensive part.

In business news, the term usually comes up in connection with cloud providers, i.e., companies that rent out computing power. Microservices are a main reason why companies use tools like Kubernetes, software for managing large numbers of containers. When a bank announces it’s modernizing its old IT, it almost always means converting a monolith into microservices.