Kubernetes

Large internet services don’t run on a single computer, but on hundreds or thousands of machines in a data center. Someone has to decide which program runs on which of these machines. That is exactly what Kubernetes does: it is software that automatically distributes programs across many machines and keeps them running there. If a machine fails, Kubernetes restarts the affected program elsewhere. If the number of users rises, it starts additional copies of the program. So you merely describe the desired state, and the software makes sure reality matches it.

From pampering servers to assembly-line work

In the past, a server was like a pet: it had a name, a human took care of it, and when it got sick, people nursed it back to health with great effort. With thousands of machines, that no longer works. Kubernetes treats machines like interchangeable parts. Broken parts get replaced, not repaired. This allows a small team to operate a very large infrastructure.

The second reason is money. Computing power in the data center costs money per hour, whether it’s being used or not. Kubernetes can scale down capacity at night and scale it back up during peak hours. This exact effect is what makes the technology attractive to companies with heavily fluctuating user numbers. For an online shop on Black Friday, the difference between normal and peak load can easily be a factor of ten.

For AI applications, there’s a third reason. Special computing chips for AI, known as GPUs, are expensive and scarce. Kubernetes distributes computing tasks across these chips in such a way that as little capacity as possible sits idle. That’s why this software is found today in almost every larger AI data center.

Containers, pods, and the desired state

The foundation is containers. A container is a program together with everything it needs to run, packed into a single standardized package. Think of a shipping container: the crane doesn’t care about the contents, only about the standardized dimensions. This way, the same package can start on any machine without anyone having to prepare it beforehand. Kubernetes groups one or a few containers together into a unit called a pod and treats this unit as the smallest building block.

The operator then writes a kind of order slip. It states, for example: five copies of this pod should always be running, and each may use at most one gigabyte of memory. This slip is sent to the system’s control center, the control plane. A component called the scheduler looks for a suitable machine with free capacity for each pod.

After that, an endless loop runs. Kubernetes constantly compares the ordered state with the actual state. If only four copies are alive, it starts a fifth. If a pod stops responding to health checks, it is terminated and replaced. This principle is called the reconciliation loop and is the core of the entire software.

Where K8s shows up in cloud bills and job postings

As a user, you never see Kubernetes directly. Indirectly, you use it almost every day, since streaming services, banking apps, and chatbots typically run on it. The software originally came from Google and was released as an open-source project in 2014. Today, a foundation, the Cloud Native Computing Foundation, maintains the code, and no one owns it alone.

In the news, the term is usually abbreviated to K8s and appears in connection with cloud providers. Amazon, Microsoft, and Google all sell ready-made Kubernetes services where the customer doesn’t have to worry about the control center. For companies, this is an argument against being locked into a single provider: the same containers will, in principle, run anywhere. In job postings, the term almost always appears where DevOps or platform engineering roles are being sought.

A common misconception is that Kubernetes is the same as Docker. Docker is primarily the tool used to build containers and start them individually. Kubernetes manages very many such containers across many machines. A second misconception: the software isn’t always worth it. For a small website, the effort clearly outweighs the benefit.