AI Chip

Every computer has a main processor. It is an all-rounder: it can do almost anything, but only a few tasks at the same time. Programs with artificial intelligence need something different. At their core, they consist of enormous amounts of very simple calculation steps, mainly multiplying and adding, which can all run simultaneously. An AI chip is a computing component tailored exactly to that. It is less versatile than a main processor, but handles this one type of work a hundred times faster.

Why the world market revolves around these components

Modern AI systems are impossible without such chips. Training a large language model, that is, having it learn from example texts, can keep thousands of chips busy for weeks. On regular main processors, the same work would take years and cost many times more electricity. Anyone who wants to develop AI must therefore first buy or rent chips.

That is why AI chips have become an economic and political issue. The company Nvidia became one of the most valuable companies in the world largely because of this. Its AI chips were so scarce for years that customers waited months for deliveries. At the same time, the USA restricts the export of the fastest models to China. A component barely larger than the palm of a hand thus helps decide the balance of power between states.

The issue is also very tangible for data centers. A single high-performance chip can cost several tens of thousands of euros and consumes as much electricity as several households. That is why new data centers are being built worldwide directly next to power plants.

Many small computing units instead of a few powerful ones

The decisive trick is called parallelism. A main processor has perhaps eight to sixteen powerful computing cores. An AI chip, by contrast, has thousands of very simple computing units. Each one alone is weak, but all of them work simultaneously on the same large computational block. You can picture it like a class splitting up a long addition list: thirty students are faster than a single math teacher.

The second trick concerns the precision of the numbers. AI models do not need highly precise decimal numbers. Rough numbers with few digits are usually enough, and the coarser the numbers, the more of them fit through the chip at the same time. Many AI chips therefore deliberately calculate less precisely than a pocket calculator.

The third point is memory, and this is often where the bottleneck lies. The computing units are so fast that they end up waiting for data. That is why, in modern AI chips, the memory sits directly next to the computing part, within the same housing. A common misconception is that computing power alone is what counts. In practice, it is often memory bandwidth, that is, how quickly data can be supplied, that determines performance.

From the data center to the smartphone

The best-known AI chips are found in data centers. That is where services like ChatGPT, image generators, or translation systems run. Originally, these were graphics cards, that is, chips for computer games, because image rendering requires the same kind of parallel computing. Today, many of them are no longer even connected to screens.

You probably carry smaller AI chips around in your pocket. In smartphones, the component is often called an NPU, short for neural processing unit. It ensures that face recognition, voice dictation, or photo enhancement run directly on the device. This is faster and better protects data, because nothing is sent over the internet.

In business news, you usually encounter the term in three contexts: quarterly results of chip manufacturers, export restrictions, and investments in data centers. An important distinction is that chips are designed and manufactured separately. Nvidia designs them, while manufacturing largely takes place at the Taiwanese contract manufacturer TSMC. It is precisely this dependency that makes Taiwan a politically sensitive location.