Compute Allocation

Large AI programs run on specialized computing chips housed in massive data centers. In the industry, this computing power is simply called “compute.” It is limited: a company owns or rents a certain number of such chips, and in the short term there are no more to be had. Compute allocation is the decision about which team or which project gets how much of it. You can think of it like distributing a school budget: the money is fixed, so someone has to decide whether it goes into new computers, into books, or into a class trip. At AI companies, this isn’t about thousands, but about billions of euros per year.

Why computing power became the scarcest resource

Until a few years ago, computing power was a minor line item for most software companies. In AI, it has become the largest cost block. A single modern AI chip costs a five-figure sum to purchase. A training run for a large language model can keep tens of thousands of such chips busy for months. The resulting bill quickly reaches into the hundreds of millions.

On top of that, you can’t simply reorder these chips. There are only a few manufacturers worldwide, and delivery times often run to many months. Power and cooling are also a bottleneck, since a large data center consumes as much energy as a small city. So anyone who wants more compute today cannot solve the problem with money alone.

That’s why the distribution of compute has become a question of power. A research team without allocated computing power simply cannot try out its idea. At several AI companies, disputes over exactly this allocation have been a reason for well-known researchers leaving. Allocation thus indirectly decides which research directions even get a chance.

How a company divides its compute budget

Compute is usually split into a few large pools. One pool goes into training, meaning the months-long process of teaching new models. A second goes into ongoing operations, when users ask the finished model questions. A third, smaller pool remains reserved for experiments and safety testing.

These pools compete directly with one another. Whoever puts more chips into training has less left over for customers. This is exactly why providers sometimes introduce response limits or delay new features during a surge in demand. The computing power was needed elsewhere at that point.

Technically, allocation runs through management software known as schedulers. These programs maintain a queue and assign each job a fixed number of chips and a time span. Teams receive quotas, similar to how a phone plan has a data allowance. Once the quota is used up, you have to wait or ask internally for more.

Where the term appears in business news

In the quarterly reports of large technology corporations, compute allocation is a recurring topic. There you’ll find how many billions are flowing into new data centers and which areas get priority. Contracts between cloud providers and AI companies also revolve around this: a provider guarantees a partner a fixed amount of computing power over several years. Such commitments now move stock prices.

A common misconception is to equate compute allocation with the purchase of chips. What’s actually meant, however, is the distribution of already existing capacity, not its procurement. A related term is compute budget: it refers to the amount of computing power planned for a single project. Allocation is the overarching decision about who receives this budget in the first place.

The principle also appears outside of corporations. Universities distribute computing time on their high-performance computers to research groups through applications. And anyone using an AI tool as a private individual feels the effects of allocation indirectly: through wait times, message limits, or the fact that a paid version responds faster than the free one.