TPU

A TPU is a computer chip that Google built to run artificial intelligence calculations. The abbreviation stands for Tensor Processing Unit. A tensor is nothing mysterious here, but simply a large table full of numbers. It is exactly these kinds of tables that an AI system multiplies billions of times over when it generates a sentence or recognizes an image. A regular main processor in a laptop can do this too, but agonizingly slowly. The TPU, by contrast, can do almost nothing else — but it handles this one task extremely quickly and with little power.

Why Google built its own chip

Around 2013, Google calculated what would happen if all users used voice recognition for just a few minutes each day. The result: the number of data centers would have had to double. Instead of simply buying more hardware, Google designed its own chip. The first TPU ran internally in 2015 and was publicly unveiled in 2016.

Economically, this remains the more exciting point to this day. The market for AI chips is almost entirely dominated by a single company: Nvidia. Anyone buying from them pays high prices and often waits months for delivery. Google is one of the few corporations that has made itself partly independent of this. That’s why the term TPU regularly turns up in stock market news, usually in connection with the question of just how secure Nvidia’s lead really is.

Other companies are taking this path as well. Amazon has chips called Trainium and Inferentia, Microsoft is building Maia. The technical term for this is custom silicon: self-designed chips built for one’s own needs. The TPU is the oldest and most mature example of this.

Computing with an assembly line of multipliers

The heart of a TPU is a grid made up of thousands of tiny computing units that can only multiply and add. The numbers are fed in on one side and move step by step through the grid, with each cell passing an intermediate result on to the next. Experts call this a systolic array. You can picture it like an assembly line: no worker has to run to the warehouse in between, the workpiece moves on to the next station by itself.

This is exactly where the trick lies. In conventional processors, most of the power isn’t spent on computing itself but on shuttling data back and forth between memory and the processing unit. The TPU avoids most of these detours. On top of that, it deliberately computes less precisely than a normal processor, often using heavily shortened number formats. For AI, this is enough, because what matters there are trends, not the twelfth decimal place.

A common misconception is that a single TPU is some kind of miracle device. What actually matters is quantity. Google wires together thousands of chips into so-called pods, connected via their own high-speed links. Only this combination trains large language models within a reasonable amount of time.

Where TPUs quietly run in the background

You cannot buy a TPU directly. It only exists inside Google’s data centers and is rented out by the hour via Google Cloud. Anyone using Gemini, querying Google Search, or getting help from Google Translate is indirectly using TPUs. The AI company Anthropic has also announced that it will deploy Google’s chips on a large scale.

In the news, you’ll usually encounter this term in two contexts. First, with new chip generations that Google unveils each year under names like Trillium or Ironwood. Second, in analyses of whether Nvidia is losing market share. Incidentally, a smaller relative also exists in phones: Pixel smartphones carry a chip based on a similar idea that computes AI functions directly on the device.