Edge

When you edit a photo or dictate a voice message, some device has to do the computing. This computational work can happen in two very different places. It can take place in a large hall full of computers owned by a company and located somewhere in the world. Or it can happen directly in the device you’re holding in your hand. This second case is exactly what’s called Edge, meaning “edge” or “rim”: the outer edge of the network, where the devices and the people are. The counterpart to this is called Cloud, meaning the central computing halls in the background.

Why the edge of the network is suddenly interesting

The first reason is time. Sending data to a data center and retrieving the response back takes milliseconds to seconds. For a chatbot, this doesn’t matter. For a car detecting a pedestrian, it does matter. A braking process must not depend on whether the mobile network happens to be cooperating.

The second reason is privacy. What the device itself processes doesn’t need to be sent anywhere. The facial recognition that unlocks your phone therefore runs locally. Your face isn’t stored on a foreign server. Companies now actively advertise this, because many users specifically ask about it.

The third reason is cost and bandwidth. A surveillance camera generates video data around the clock. Uploading all of it to the internet would be expensive and mostly pointless. It makes more sense to let the camera itself decide when something noteworthy happens. Only then does a short clip go outside.

How small chips handle large models

A phone has far less computing power and memory than a data center. Large AI models don’t fit in there without further ado. That’s why they get shrunk. A common method is called quantization: the numbers in the model are stored more coarsely, for example with eight instead of sixteen digits of precision. This shrinks the model significantly while it loses only a little quality.

A second approach is distillation. In this process, a large model teaches a small one to give similar answers. The small model is then not an all-rounder, but it handles its one task well enough. On top of that comes specialized hardware. Almost every new smartphone contains its own chip area just for AI calculations, often called an NPU.

In practice, manufacturers mix both worlds. The device answers simple queries itself, and sends difficult ones to the cloud. This division is called hybrid. Ideally, the user notices nothing of this, except that some answers work even without internet.

From smartwatch pulse to factory floor

You encounter Edge constantly in everyday life, without it being called that. The keyboard’s autocorrect runs locally. So does your headphones' noise cancellation. A smartwatch detects heart rhythm disturbances on the wrist, not on a server. Even robot vacuum cleaners map the home using their own computing power.

In industry, the term is more common. Machines in factories monitor their own vibrations and report wear early. Agricultural machinery distinguishes weeds from crops in the field and sprays selectively. Both require immediate decisions in places with poor network coverage.

In business news, “Edge AI” or “on-device AI” mostly appear as a growth promise. Chip manufacturers profit from this, because every device needs new hardware. A common misconception, by the way, is that Edge is simply the opponent of the Cloud. The two complement each other: large models continue to be trained centrally, while they’re increasingly used at the edge.