Neural Engine

An iPhone or Mac doesn’t contain just a single computing block, but several that share the work. One of them is called, at Apple, the Neural Engine. It is built for a very specific kind of computing task: the massive number multiplications that occur when a program recognizes a face in a photo or converts spoken language into text. A device’s main processor can also solve such tasks, but needs more time and more power to do so. The Neural Engine can do almost nothing else, but it does this one thing very well. It has been part of every iPhone chip since 2017, and other manufacturers now build similar blocks under the name NPU.

Why AI gets its own dedicated block

A main processor is an all-rounder. It can calculate spreadsheets, render web pages, manage files. This flexibility costs space on the chip and energy during operation. A specialized block forgoes everything it doesn’t need, and thereby becomes significantly more efficient at its one task. For AI calculations, the differences are often tenfold or more.

In a smartphone, energy consumption decides everything. A battery has only a few watt-hours, and a hot chip throttles itself. It is only because the Neural Engine works efficiently that the camera can run several AI steps in the background for every photo without the battery noticeably suffering.

The second reason is privacy. If the calculation happens on the device, the photo or voice recording doesn’t need to be sent to a server on the internet. Apple has been advertising exactly that for years. For users, this means: some features work even in airplane mode.

What actually happens on the chip

An AI model is, at its core, a very long chain of multiplications and additions. These are called matrix multiplications, because entire tables of numbers are computed together. In image recognition, there are billions of such computational steps per photo. All the steps have the same structure, and many of them can be executed simultaneously.

The Neural Engine exploits exactly that. It consists of many small, identical computing units that work in parallel. You can imagine it as an assembly line: instead of one versatile craftsman, there stand hundreds of workers, each of whom performs only one single motion. As long as the task fits that one motion, the assembly line is unbeatable.

Additionally, the Neural Engine deliberately calculates with less precision. It stores numbers more coarsely, often with only eight or sixteen digits in binary instead of thirty-two. For AI models, this is usually enough, because small rounding errors barely change the result. It’s important to distinguish this from training: the Neural Engine applies models that have already been fully trained; it doesn’t teach them anything new. The heavy learning still happens in data centers on graphics cards.

From the Photos app to chip marketing

In everyday use, you never notice the Neural Engine directly, only through features. The Photos app sorts images by people and pets. The keyboard suggests the next word. You can cut a subject out of an image by tapping on it. Live captions and translations work without a connection. All of these are models that compute on this block.

In product presentations, the Neural Engine is a marketing argument. Apple then cites figures like 38 trillion operations per second. Such figures sound precise, but are hard to compare because every manufacturer counts differently. A common misconception, moreover, is that the Neural Engine is a separate chip. It is an area within the main chip, right next to the processor and the graphics unit.

Economically, the trend is interesting. Qualcomm, Intel, and AMD now also build in such units and advertise with the term AI PC. Behind this lies the question of where AI will compute in the future: in expensive data centers or on the device in your hand. The stronger these blocks in the device become, the more the work migrates there. Large language models, however, remain too large for a phone for the time being.