Spatial Computing

A normal computer knows the world only as a surface: a window, a mouse pointer, a screen. Spatial computing breaks with this surface. Devices such as glasses with built-in cameras continuously measure the room in which one stands. They know where the wall is, where the table stands, and where one is currently looking. On this basis, they can display images so that they appear anchored in space. A video then seems to hang on the wall and stays there, even if one turns around and comes back.

Why the room as a screen is interesting

The screen of a laptop is limited. One can only see a few windows at once and constantly pushes them on top of each other. A room, by contrast, is large. Whoever can distribute digital content within it suddenly has a great deal of working space. This is exactly what manufacturers of such glasses advertise: several large windows at once, without buying a single monitor.

The second reason lies in learning and working. Some things are better understood when one can walk around them. A car engine as a spatial model in the workshop is more illustrative than a drawing. Surgeons practice procedures on virtual bodies, technicians see repair instructions lying directly on the machine. In such cases, the technology saves time and prevents errors.

Economically, spatial computing is considered a possible successor to the smartphone. Whether this will happen is open. So far, the devices are expensive, heavy, and uncomfortable for many people. Nevertheless, the term appears in almost every report about the future of Apple, Meta, or Google.

How a device understands the room

First comes the measurement. Cameras and sensors capture distances and surfaces and build a digital image of the room from them. Experts speak of a map of the environment. The device then knows that a wall stands two meters ahead and a table in the middle. Motion sensors additionally report how the head turns and tilts.

Then comes the recognition. Here artificial intelligence works, that is, software that has learned from many example images to distinguish things. It separates floor from wall, recognizes hands, and tracks the direction of gaze. That is why modern glasses often no longer need a controller. One looks at a window and taps thumb and index finger together.

Finally comes the display. The image for the left and right eye is calculated slightly offset, creating depth. Speed is crucial: only a few thousandths of a second may pass between head movement and the new image. If the delay is greater, the world appears sluggish and many people feel nauseous. This computational work is the reason why such glasses become warm and the battery rarely lasts longer than two to three hours.

From the Apple Vision Pro to the furniture app

The best known are head-mounted devices. Apple deliberately used the term spatial computing in 2023 for the Vision Pro, avoiding the older word virtual reality. Meta sells significantly cheaper glasses with its Quest series. Both show the real environment via cameras or transparent lenses, supplemented with digital objects.

However, one also encounters the technology without glasses. Furniture stores offer apps that project a wardrobe true to scale into one’s own living room. Navigation apps overlay directional arrows onto the camera image of the street. This weaker form is called augmented reality. Spatial computing is the broader term and also refers to robots or cars that spatially capture their surroundings.

A common misconception: spatial computing is the same as the metaverse. The metaverse describes a shared virtual world, that is, a place. Spatial computing, by contrast, describes a way of interacting. One can work spatially with a spreadsheet, entirely without foreign avatars. Anyone reading reports on this topic should therefore pay attention to the difference.