NPU

Every computer has a main processor. It’s an all-rounder capable of handling almost any computing task, but not especially well at any single one. That’s why manufacturers add specialized chip components that only handle one type of task, but do so very quickly. An NPU is one such specialized component, specifically for programs that have learned from examples — in other words, for artificial intelligence. The abbreviation stands for Neural Processing Unit, meaning roughly a "unit for processing neural networks." A neural network, in this context, is a computational model made up of many simple calculation steps arranged in layers, loosely inspired by the brain.

Why AI gets its own chip space

AI programs compute differently than a spreadsheet does. They multiply huge lists of numbers with one another and add up the results — millions of times, always following the same pattern. An all-round processor tends to work through such tasks one after another. That’s exactly why it takes a long time and consumes a lot of power.

On a phone, this is a real problem. The battery is small, and a chip that gets too hot throttles itself. An NPU accomplishes the same task using a fraction of the energy. Manufacturers often claim that their NPU handles AI tasks ten times faster or more than the main processor, at the same power consumption.

There’s a second reason as well: privacy. If the NPU is powerful enough, a voice command or a photo no longer needs to be sent to a server on the internet. The computation happens directly on the device. This is called on-device AI, and it’s one of the most important selling points for NPUs.

What’s wired differently inside an NPU

An NPU consists of a very large number of small computing units that work in parallel. Each one can do only a little: multiply, add, pass the result along. But they all do it simultaneously. You can picture this like a factory floor with a thousand workstations, each repeating the same simple motion over and over.

A second trick lies in the precision of the numbers. A normal processor calculates with very precise numbers, often using 32 digits in binary format. AI models usually get by with just 8 digits, sometimes even fewer. Shrinking the numbers this way is called quantization. Smaller numbers mean less memory needed and more calculations per second.

An important distinction: NPUs generally don’t train AI models. Training means letting a model learn from examples, and that happens in large data centers on graphics chips. The NPU only handles using the finished model. Another common misconception is that NPU performance alone determines a device’s AI capabilities. Often, it’s the memory that acts as more of a bottleneck than the chip.

NPUs in phones, laptops, and marketing brochures

Almost every new smartphone contains an NPU, even though it goes by different names. Apple calls it the Neural Engine, Google refers to it as the Tensor chip, and Qualcomm calls it a Hexagon NPU. It’s at work when a phone recognizes faces in photos, filters out background noise during calls, or converts spoken language into text.

Since 2024, the term has also been appearing in the PC market. Microsoft has set a minimum NPU performance requirement for the “Copilot+ PC” label: 40 TOPS, meaning 40 trillion operations per second. This number has since appeared on many laptop spec sheets. However, it’s a rough benchmark and says relatively little about how fast a particular program will actually run.

In business news, you’ll mostly encounter this term in connection with chipmakers. When Apple, Qualcomm, or Intel unveil new processors, NPU performance is now the figure that gets mentioned first. For investors, it’s of interest because it shows how much AI computation will happen in devices in the future, rather than in data centers.