LazyAR

Some devices overlay digital content onto the image of the real environment. Glasses, for example, show an arrow on the sidewalk, a phone shows a name above a building. For this to look believable, the device must constantly know where it is and where it is looking. Normally, it recalculates every single frame completely, up to 60 or 90 times per second. LazyAR breaks with this habit: it deliberately computes lazily, meaning only when and only as much as the situation actually requires. The English word “lazy” means exactly that – lazy in the sense of “deferring”.

Why battery and heat are the real obstacle

Glasses sit on your head. They must neither be heavy nor get hot. This leaves very little room for battery and cooling. That is exactly why many glasses run out of power after one or two hours. The computational work for the overlaid content is one of the biggest power drains here.

There is a second problem on top of that: latency. If you turn your head and the overlay only catches up 100 milliseconds later, it no longer feels like part of the world. Some people even get nauseous from this. Saving computation time therefore means not only longer battery life, but also faster response.

LazyAR builds on a simple observation. Between two consecutive frames, almost nothing usually changes. A table is still standing in the same spot after 16 milliseconds as it was before. Recalculating everything anyway is wasted energy.

What the device skips and what it reuses

The core of the method is a distinction between expensive and cheap tasks. Expensive, for example, is object recognition, that is, the question: What can actually be seen in the image? Cheap, on the other hand, is shifting an already existing result. If the head has only moved slightly, the device shifts the old overlay accordingly. Only once the scene changes significantly does the expensive computation run again.

For this to work, a watchdog is needed that measures the change. Rough figures are often enough for this: How much has the viewing angle rotated? How many pixels look different than a moment ago? If this value exceeds a threshold, recomputation happens. If it stays below, the old result continues to apply.

An analogy helps: a weather service doesn’t measure the temperature anew every second, but every few minutes. In between, the last measured value applies, because temperature changes slowly. During a thunderstorm, measurements are taken more frequently. LazyAR does the same thing with images. The typical misconception here: lazy does not mean inaccurate. The thresholds are set so that people are not supposed to see the difference.

In glasses, navigation apps, and phone cameras

You notice the effect most clearly with smart glasses, such as those sold by Meta, Apple, or Snap. There, this kind of frugality determines whether a device lasts two hours or half a day. Manufacturers rarely name the technology explicitly. In technical articles and at developer conferences, however, it comes up regularly.

The idea is also present on phones. When a navigation app places arrows on the road, or a furniture app puts a sofa in your living room, the same trade-off is running in the background. Something similar is known from games: there, an image is sometimes computed at a lower resolution and then upscaled.

LazyAR should be distinguished from offloading computation to a data center. This also saves power on the device, but causes latency over the network. LazyAR, by contrast, works locally and avoids work instead of shifting it elsewhere. In practice, manufacturers combine both approaches.