Cluster (Compute Cluster)

A compute cluster is a combination of many individual computers that jointly work on a task. The individual machines are called nodes. They typically sit as flat boxes in metal racks inside a large, air-conditioned building. They are connected via a very fast network so they can constantly exchange intermediate results with one another. From the outside, the whole thing appears as a single, extremely powerful machine. Anyone who logs in over the internet doesn’t see a hundred computers, but simply submits a job.

Why a single machine isn’t enough for AI

Training a large language model consists of an unimaginable number of simple computational steps. A good gaming PC wouldn’t take weeks for that, but many centuries. Nobody has that kind of time. So the work is distributed across thousands of compute cores working in parallel. A cluster is the practical answer to a pure time problem.

That’s why clusters are also an economic factor in the AI industry. The specialized chips used often cost several tens of thousands of euros apiece. A large training cluster can therefore mean investments in the billions. On top of that come electricity costs, since such a facility consumes as much energy as a small city. When the news talks about AI spending, this is usually exactly what’s meant.

However, clusters aren’t just used for AI. Weather forecasting, climate models, crash simulations in the automotive industry, and calculations in physics also run on such systems. The term is thus older than the current AI boom. What’s new above all is how strongly these facilities are now being tailored to a single purpose.

How work is distributed across the nodes

Central to this is a program that manages and assigns jobs. It accepts requests, places them in a queue, and assigns them to available nodes. Experts call such a program a scheduler. You can think of it like the dispatch office of a trucking company. It decides which job goes onto which available truck.

During AI training, the nodes don’t work on separate tasks but on the same one. Each one computes with a different portion of the training data and must then reconcile its results with all the others. This reconciliation happens thousands of times per training run. That’s why the network between the nodes is often more important than the compute power itself. If it’s too slow, expensive chips sit idle waiting for data.

A second fundamental principle is fault tolerance. With thousands of components, something fails somewhere almost every day, statistically speaking. A cluster is therefore built so that the failure of one node doesn’t halt operations. Training runs regularly save intermediate states and resume from there after an error. This is precisely what distinguishes a cluster from a single large mainframe computer.

Clusters in headlines and in the cloud

Most often, one reads about clusters when corporations announce new data centers. Then figures like “100,000 graphics chips” or numbers on power consumption in megawatts come up. These figures are now regarded as a measure of how seriously a company is competing in the AI race. Location debates also revolve around this, for instance when residents complain about water and energy demands.

But you don’t need to own your own cluster to use one. Large providers rent out compute time by the hour over the internet — this is called the cloud. A research group or a startup then books a few nodes for a project and pays only for the hours used. Universities also operate their own clusters for students and researchers.

A common misconception is that cluster and cloud are the same thing. Cluster describes the technical architecture, cloud the business model behind it. Similarly, cluster is sometimes confused with supercomputer. Most of today’s supercomputers are simply very large, especially tightly coupled clusters.