Full-Stack Infrastructure

For a chat program like ChatGPT to be able to respond, a great many things have to work together. It needs special chips that do the computing work. It needs halls full of computers that house these chips, cool them, and supply them with power. It needs programs that distribute the computing work across the chips. And at the very top it needs the actual AI program along with the app the user sees. These layers stacked on top of each other are called a stack, i.e. a pile. We speak of full-stack infrastructure when a single company owns or at least controls all of these layers itself, rather than buying individual ones in.

Why corporations want the whole stack

The most important reason is money. Whoever owns only one layer has to give something up to everyone else. A provider that sells an AI app but has to rent chips passes on a large share of its revenue to the chipmaker. Whoever controls the entire stack, by contrast, keeps the profit margin of every layer for itself.

The second reason is dependency. Graphics chips for AI were scarce for years, and a single manufacturer, Nvidia, dominated the market almost completely. Whoever was waiting for deliveries could not train their models. Google and Amazon therefore developed their own AI chips in order to be less vulnerable to pressure. At Google such chips are called TPU, at Amazon Trainium.

The third reason is speed. When chip, data center, and software all come from the same house, they can be tuned to one another. If a chip’s memory matches exactly what one’s own model needs, that saves computing time. This fine-tuning is difficult when three outside companies are involved that won’t let each other see their cards.

The layers of the stack from bottom to top

At the very bottom lies the hardware. This includes the computing chips, but also the networking technology that connects thousands of chips into a single large machine. These connections are not a minor detail. During the training of a large model, the chips constantly exchange intermediate results, and a slow connection slows down the entire setup.

Above that lies the data center itself: the building, the power supply, the cooling. A modern AI facility can require as much electricity as a small city. Some operators therefore buy entire power plants or sign long-term contracts with energy suppliers. This too now counts as part of the stack.

The next layer is the software that makes the hardware usable. It breaks a computing task into parts and distributes them across the chips. Nvidia’s software package CUDA is the best-known example here and a main reason why the company is so hard to replace. At the very top come the trained model and the application that customers work with.

Who owns the full stack and who doesn’t

Google currently comes closest to the ideal. The corporation builds its own chips, operates its own data centers, develops its own software, and sells its own model with Gemini. Nvidia is taking the same path from the bottom up: first chips, then networking technology, then software, and by now even complete data center designs. Amazon and Microsoft are also involved, but continue to buy in large quantities of chips.

In the news, you mostly encounter the term in connection with quarterly earnings or billion-dollar investments. When a company announces that it is now also building its own chips, that is a step toward full stack. Analysts often assess such steps positively because they lower costs in the long run. In the short term, however, they cost an enormous amount of money, since chip development takes years.

A common misconception is that full stack is automatically better. Whoever does everything themselves ties up a great deal of capital and risks being weaker in each individual discipline than a specialist. Smaller AI companies therefore deliberately rely on rented computing power and focus solely on their model. Both strategies exist side by side, and which one pays off depends above all on the size of the company.