Cloud Deployment

When you use an app, the program has to run somewhere. In the past, this was almost always the machine in front of you. Today, the actual work often happens in a data center: a hall with thousands of computers, reachable over the internet. Such halls belong to companies like Amazon, Microsoft, or Google, and computing power there is rented by the hour. This exact process of setting up and operating a program on rented machines is called cloud deployment. Your device then merely sends the request and displays the answer.

Why AI can hardly do without a data center

Large AI models need special chips, so-called graphics processing units, or GPUs for short. A single such chip for data centers often costs more than a small car. For a modern language model, you don’t need just one, but many of them at once. No ordinary user buys this kind of equipment. So a company deploys the model once in the cloud, and millions of people share this hardware.

For companies, the second advantage is the payment model. You don’t buy devices, but pay for the time you use them. A start-up can thus start with ten users in the morning and serve ten thousand by evening, without buying hardware beforehand. This ability to grow along is called scalability. Conversely, costs go back down again when demand drops.

That’s why cloud deployment is also an important topic on the stock market. The cloud divisions of Microsoft, Amazon, and Google are among their most profitable business areas. When an AI company grows, these providers earn along with it, because the computing time is rented from them.

From program code to a running service

The process is usually similar. First, the program, along with everything it needs to run, is packaged into a self-contained package. Such packages are called containers, because they work the same everywhere, like shipping containers. Then the package is started on rented machines. Management software monitors whether enough machines are running and automatically starts new ones if there are failures.

In front of this sits a load balancer that distributes incoming requests evenly across the machines. This way, no single machine gets overwhelmed. With AI services, there’s also a queue, because computing an answer takes seconds, not milliseconds. As requests increase, additional machines are switched on. As they decrease, machines are switched off and stop costing anything.

A common misconception is that the cloud is a single place, the same everywhere. In reality, the data centers stand at concrete addresses, and this has consequences. A service in Frankfurt responds faster to users in Europe than one in California. Additionally, European data protection law often requires that data belonging to EU citizens stay in Europe.

Cloud, your own server room, or a hybrid form

In everyday life, you constantly encounter cloud deployment without noticing it. A chatbot in the browser, photo search on your phone, voice messages being converted into text: the computing work almost always happens in a data center. Your phone, in this case, is mainly a display device and microphone. Without an internet connection, such features therefore don’t work.

The counter-variant is called on-premise: the company sets up the machines itself, in its own premises. Banks, hospitals, and government agencies do this because sensitive data shouldn’t leave the building. In return, they have to buy, maintain, and replace the equipment themselves. Many choose a middle path, the hybrid cloud: sensitive data stays in-house, while computation-intensive tasks run in the cloud.

In the news, two other terms often show up alongside this one. Edge computing means that a small model computes directly on the device, such as facial recognition on a phone. And vendor lock-in describes the dependency on a single provider, when switching becomes too expensive and cumbersome. Anyone who weighs the pros and cons also understands the outage reports: when a large data center goes down, hundreds of services often go down at the same time.