Nitro Chips

Large internet companies operate huge halls full of computers, so-called data centers. There, customers rent computing power instead of buying their own machines. Inside Amazon's servers, alongside the normal main processor, there is additional proprietary hardware: the Nitro chips. These components handle the work surrounding the actual computing job, namely sending data over the network, reading and writing to disks, and controlling who is allowed to access what. Amazon designed them itself and has been using them in its cloud division AWS since 2017. A single server usually contains several such cards, each with different tasks.

What Amazon gains from it

In the past, the management of a server ran on the same processor as the customers' programs. This management software is called a hypervisor. It splits a physical machine into several virtual machines that feel independent of one another. This used to cost a noticeable amount of performance: estimates spoke of around 30 percent being consumed by management tasks. Amazon could not sell that computing time.

Nitro shifts this work onto dedicated chips. As a result, the main processor belongs almost entirely to the customer. Across hundreds of thousands of servers, this is an enormous economic lever. Amazon can offer lower prices or earn more, without buying more hardware.

Then there is security. Because management runs on separate hardware, it has no access to customer data whatsoever. Even Amazon employees do not have easy access to running customer systems. For banks, hospitals, and government agencies, this is precisely one argument for moving to the cloud in the first place.

Division of labor within the server

A Nitro server consists roughly of three parts. First, the Nitro cards: plug-in cards with their own chips that handle networking and storage. Second, a very lightweight hypervisor that now only distributes the compute cores and otherwise does almost nothing. Third, a security chip that checks at power-on whether the software on the server is unaltered.

You can picture it like a restaurant. The cook is the main processor and is meant to do nothing but cook. Purchasing, billing, and door control are handled by separate staff with their own rooms. The cook is not bothered by any of this and is never interrupted. And whoever stands at the door never enters the kitchen.

It is important to distinguish Nitro from other Amazon chips. Nitro does not itself perform computations for AI models. For that, AWS has the chip families Trainium for training and Inferentia for answering queries. Nitro merely ensures that these specialty chips are supplied with data cleanly and quickly.

Nitro in price lists and quarterly earnings

Nitro rarely becomes directly visible, since nobody buys a Nitro chip in a store. But anyone who rents a server from AWS practically always uses it. Almost all current server types at AWS are based on this technology. In technical data sheets, it appears as the explanation for high network speeds.

In business news, Nitro appears as an example of a trend: major cloud providers are increasingly developing their own chips instead of buying them. Google, Microsoft, and Meta are doing similar things. For suppliers like Intel or Nvidia, this means less negotiating power in the long run.

A common misconception is that Nitro makes servers faster in the sense of more powerful processors. That is not true. The compute cores remain the same; they are simply no longer slowed down by side tasks. The gain lies in the division of labor, not in more raw performance.