Compute Capacity

Every computer works in small steps. It adds numbers, compares them, moves them around in memory. Compute capacity indicates how many such steps are possible in a given amount of time. You can think of it like the capacity of a factory: not how well it works, but how much it can produce per day. Anyone who wants more capacity needs more machines, or faster ones. In the AI industry, the English word “compute” is often used, and it means exactly the same thing.

Why compute became a scarce resource

Modern AI systems learn from enormous amounts of data. This learning consists of trillions of individual computational steps. For a large language model, tens of thousands of specialized chips run in parallel for weeks to accomplish this. Without this amount of compute capacity, such a model would be technically impossible, no matter how clever the underlying idea is.

That’s why compute capacity is now an economic issue, not just a technical one. The relevant chips are called GPUs, or graphics processing units, which can carry out many calculations simultaneously. They are expensive and sold out for months at a time. Whoever owns a lot of them can try things that remain out of reach for smaller companies. This is exactly why corporations like Microsoft, Amazon, or Meta purchase chips worth tens of billions of dollars.

Even nations now treat compute capacity as a strategic asset. The US restricts the export of the fastest AI chips to China. Europe is building its own data centers so as not to be completely dependent on American providers. This shows that compute has become a factor of power, similar to oil or semiconductors in general.

How compute capacity is measured and combined

The most common unit is FLOPS, short for floating-point operations per second. A good laptop chip can perform a few trillion of these. An AI data center achieves many times that, because thousands of chips work together there. For entire training runs, the measure used is not FLOPS per second but the total number of all operations. For the largest models, this figure is around 10 to the power of 25 to 10 to the power of 26.

However, the capacity of a data center is not simply the sum of its chips. The chips must constantly exchange their intermediate results. If the connection is too slow, they wait for each other and sit idle. That’s why fast networks within a data center are almost as important as the chips themselves.

Then there’s the electricity. A large AI data center consumes as much energy as a small city, and the waste heat has to be cooled away. Some planned facilities fail not because of a chip shortage, but because the local power grid can’t supply enough. Compute capacity therefore always has a physical limit.

Where compute shows up in prices and headlines

The most direct encounter with this topic is in pricing. Free chatbots often respond using smaller, more economical models. Paid versions draw on more compute-intensive ones, which are allowed to “think” longer. When a provider introduces usage limits, it’s almost always due to insufficient compute capacity.

In financial news, compute capacity is the reason behind the rise of chipmaker Nvidia. Its graphics processing units are considered the standard for AI training, which has made the company one of the most valuable in the world. Reports about new data centers, power contracts, or chip orders now regularly move entire stock markets.

A common misconception is that more compute automatically makes a model better. Compute capacity is only one of three prerequisites, alongside good data and a good model design. It sets an upper limit on what’s feasible, not a guarantee of quality. Anyone who pours a lot of compute time into a poorly built model simply burns through money faster.