Compute Cluster

A single computer has limited computing power. But some tasks are so large that one device would need decades to complete them. The solution: you put many computers in a hall, connect them with very fast cables, and let them work on the same task together. This assembly is called a compute cluster. The individual devices within it are called nodes. From the outside, a cluster looks to the user like a single, very large machine.

Why clusters determine AI progress

Modern AI systems learn from vast amounts of text. This learning process is called training, and it consists of an unimaginable number of simple computational steps. A modern gaming PC would need many thousands of years to do this. A large cluster accomplishes the same work in weeks. So without a cluster, you simply cannot compete when it comes to the largest models.

That is why clusters have become an economic power factor. Companies like Microsoft, Google, Meta, or Amazon together spend several hundred billion dollars annually on data centers. A single large AI cluster can cost several billion dollars. Construction often takes one to two years, because land, power connections, and cooling all have to be built first.

Politics is also interested in this. The United States restricts the export of particularly fast chips to China, because they could be used to build large clusters. Computing power is thus treated like a strategic resource. In the news, you often see the English word compute used for this—simply meaning “computing power.”

From a single chip to a computing hall

The heart of an AI cluster is the graphics chip, or GPU for short. GPUs were originally developed for video games. Their strength lies in performing many similar calculations in parallel, and that is exactly what AI training requires. Several GPUs sit together in a server, several servers in a cabinet called a rack. A large cluster consists of thousands of such racks.

The decisive part, however, is not the calculation itself but the connection. During training, each chip processes only part of the task. After every step, all chips must exchange their intermediate results. Special networks are used for this, transferring several hundred gigabytes per second. If this connection is too slow, expensive chips simply sit idle.

Then there is power and cooling. Large clusters consume several hundred megawatts, as much as a small city. The waste heat is increasingly carried away using water, because air is no longer sufficient. A management program, the scheduler, also distributes jobs across available nodes. It is also important to distinguish clusters from the cloud: cloud describes renting computing power. A cluster is the hardware itself that stands behind it.

Clusters with names: Colossus, Stargate, and the university machine

In business news, individual clusters are now appearing under their own proper names. Elon Musk’s AI company xAI built a cluster called Colossus in Memphis with over a hundred thousand GPUs. OpenAI and its partners are pushing forward data centers worth hundreds of billions of dollars under the name Stargate. Such announcements noticeably move the stock prices of chipmakers like Nvidia.

Clusters are encountered outside of AI as well, though. Weather forecasts, climate models, and crash simulations in the automotive industry all run on them. Universities operate their own clusters for research groups. The fastest facilities in the world are called supercomputers and are compared twice a year in a ranking list.

Anyone who needs computing power today rarely buys hardware. Instead, you rent time on someone else’s cluster, often billed hourly per chip. A student wanting to train a small model uses exactly the same infrastructure as a large corporation, just for a few minutes. A common misconception, by the way, is that a cluster makes a single chatbot response faster. A single request usually only needs a few chips—the cluster mainly serves to handle very many requests simultaneously, or to handle training.