Multi-Tenancy

Multi-tenancy translates roughly to “multiple-tenant operation.” A provider runs a single piece of software on its machines, and many customers use exactly this one piece of software at the same time. Each customer sees only their own data and keeps their own settings. Technically, however, everyone shares the same servers and the same program code. A good comparison is an apartment building: one building, one heating system, one roof, but self-contained apartments. The opposite would be building a separate house for each customer — that is called single-tenancy.

Why providers put all customers in the same building

The main reason is money. Servers, maintenance, and updates cost almost the same whether ten or ten thousand customers are working on them. These costs can therefore be spread across many shoulders. This is exactly why a provider can sell software for ten euros a month that would have cost thousands as a standalone installation in the past.

The second reason is speed of updates. The provider rolls out a new version once, and by the next morning all customers have it. In the past, every company had to install updates itself, often with years of delay. That’s why multi-tenancy is the technical foundation of the SaaS business model, that is, Software as a Service: software is not bought, but rented.

The downside is dependency. If the system fails, it’s not individual customers who are affected, but all of them at once. A programming error at the provider can bring down companies worldwide. Such outages regularly make it into the business news.

How the apartments are separated from each other

Each customer gets their own identifier, often called a tenant ID. This identifier is attached to every stored data record. When the software queries data from the database, it checks the identifier along with every single request. This way, foreign data never ends up in the wrong display.

There are different degrees of strictness in how this is built. Sometimes all customers sit in the same tables and are separated only via the identifier. Sometimes each customer gets their own database, while the program code continues to run jointly. The stronger the separation, the more secure it is, but also the more expensive to operate.

A second problem is called the “noisy neighbor”: one customer starts a huge analysis and thereby slows down all the others. To counter this, providers set upper limits, such as a maximum number of requests per minute. Such limits are known from price lists as “rate limits” or quotas.

Multi-tenancy in AI services and in the cloud

Every major online service works this way. When you log in to a streaming service, a school learning portal, or a cloud storage provider, you are one tenant among millions. Even the cloud offerings from Amazon, Microsoft, and Google are based on many customers using the same hardware.

With AI, this is especially noticeable. Large language models run on very expensive graphics cards that can rarely be allowed to sit idle. That’s why the requests of many users are bundled together and sent jointly through the same model. Only this way are low prices per request possible at all.

But this is exactly where discussions about data protection arise. Companies wonder whether their confidential texts are allowed to sit on foreign servers next to those of their competitors. Some providers therefore sell more expensive variants with reserved hardware, often called “dedicated” or “private.” A common misconception, by the way, is that shared servers automatically mean the model is further trained on one’s own data. The two have nothing to do with each other and are regulated separately by contract.