NPU as a Service

For a computer to handle tasks like speech recognition or image generation, it needs to perform a huge number of simple calculations simultaneously. Ordinary main processors are poorly suited for this because they tend to work through few tasks one after another. That’s why there are specialized chips tailored precisely to this kind of mass computation: Neural Processing Units, or NPUs for short. Such chips are expensive, become outdated quickly, and often sit unused. NPU as a Service is the answer to this: a provider houses the chips in a data center, and customers book them over the internet for minutes, hours, or months. In other words, you rent computing power the way you’d rent a car instead of buying one.

Why hardly anyone buys their own AI chips

A single AI accelerator for data centers often costs several tens of thousands of euros. On top of that come power supply, cooling, networking, and personnel. A start-up with ten people can’t afford that. Through a rental service, they get the same hardware for just a few euros per hour.

The second reason is utilization. Anyone training an AI model needs enormous computing power for weeks and then almost none afterward. Owned chips would cost money during that downtime without doing anything. With the rental model, you simply switch them off and stop paying.

Then there’s the pace of development. Roughly every one to two years, a significantly faster chip generation appears. Purchased hardware loses value rapidly as a result. A provider spreads this risk across many customers and continuously swaps out the chips.

From a click to a rented computing unit

Technically, behind the offering is a data center full of servers containing the specialized chips. Management software divides these chips into portions and assigns them to customers. The customer themselves only sees a website or a programming interface—essentially a kind of socket that their own program can plug into. They upload their data and their model, start the computation, and download the result.

Separating customers from one another is crucial. Multiple companies can work on one server simultaneously without seeing each other. Virtualization ensures this: the software gives each customer an isolated environment that feels like their own computer. For security reasons, some providers also rent out entire machines exclusively.

Billing is usually done per chip-hour. Anyone using six NPUs for two hours pays for twelve chip-hours. For requests to a finished model, there is often more granular billing, for example per amount of text processed. Those planning long-term book capacity in advance and receive a discount for doing so.

NPU rental in cloud offerings and stock market news

This model is most commonly found among the major cloud providers. Amazon, Microsoft, and Google rent out AI accelerators by the hour, some from Nvidia, some of their own design. There are also smaller specialists that sell almost exclusively AI compute time. In business news, such companies are often called neoclouds.

You also encounter this principle indirectly every day. When you use a chatbot or have a photo edited by AI, the computation runs in a distant data center. In many cases, the app provider has merely rented the chips for that purpose.

A common misconception: the NPU in your phone or laptop has nothing to do with this service. It computes directly on the device, without internet, and is much weaker. NPU as a Service always refers to rented capacity in a data center. Both approaches complement each other: small tasks run on the device, while large ones are sent to the cloud.