Pod
A pod is the smallest unit you can start in Kubernetes: one or more closely related applications that share network and storage. In the AI world, "pod" also refers to a large cluster of Google computing chips that works like a single supercomputer.
Large internet services don’t run on a single computer, but on thousands of machines in a data center. To keep this manageable, programs are packed into self-contained packages called containers. A container holds a program along with everything it needs to run, and therefore behaves the same way on every machine. A pod is the wrapper around one or more such containers. Kubernetes, the software that automatically manages such fleets of machines, never starts or stops individual containers, but always whole pods. Confusingly, in reports about AI chips the same word means something different, namely a large interconnected cluster of computing chips.
Why Kubernetes doesn’t manage individual containers
Some programs belong together inseparably. A web server, for example, often needs a second small program alongside it that collects its log files and forwards them. Splitting these two parts across different machines would make no sense. The pod guarantees that they always end up together on the same machine.
All containers in a pod also share a network address and can share common storage areas. This means they can reach each other as easily as if they were running on the same laptop. Externally, however, the pod appears as a single unit. This makes the rules for networking and access permissions much clearer.
A pod is deliberately designed to be short-lived. If a machine fails, the pod is not repaired but deleted and restarted elsewhere. It’s treated like a disposable cup, not a favorite mug. Important data therefore never belongs inside the pod itself, but in a connected database or persistent storage.
From desired state to running pod
Developers describe in a text file what should run: which container image, how much memory, how many copies. This file describes a desired state, not a list of commands. Kubernetes then continuously compares the desired state with reality and reconciles any differences.
A component called the scheduler looks for a suitable machine for each new pod. It checks where enough memory and computing power are available. If a pod needs a graphics chip for AI computation, only machines with such chips are considered. A service on the chosen machine then starts the pod’s containers.
If a pod crashes, the system notices the deviation from the desired state and starts a replacement. If load increases, it can automatically spin up additional copies of the same pod. This self-healing is the actual reason for all the effort. Otherwise a human would have to get up at night to restart crashed programs.
Pods at cloud providers and in AI data centers
Almost every major online service today runs in pods, without users noticing anything about it. Amazon, Google, and Microsoft offer Kubernetes as a ready-made service. Anyone with an account there ultimately pays for the computing time their pods consume. In job postings for DevOps or cloud roles, Kubernetes experience is therefore a standard requirement.
AI models are also usually run in pods. A pod then contains the model and occupies one or more specialized chips. If a chatbot is heavily used, the provider simply starts more pods of it.
The second meaning comes up in news about data centers. Google also calls a cluster of its AI chips, the Tensor Processing Units, a pod. Such a TPU pod consists of thousands of chips connected by very fast links that jointly train a large model. This has nothing to do with Kubernetes. Which meaning is intended can be recognized from context: is it about software operations or about hardware in the data center?