Sim-to-Real Transfer

Today, robots often don’t learn in a factory hall, but in a computer simulation. This is a virtual replica of the world in which a program can try out things like walking or grasping. There, it may fail as often as it likes without anything breaking. Sim-to-real transfer is the name for the step that follows: what has been learned is transferred onto the actual machine. That sounds simple, but it is the most difficult part of the process. Because the simulation calculates with simplified rules, and reality does not stick to them.

Why robots can’t practice in the real world

Modern robot controllers are often created through trial and error. The program tries a movement, receives an evaluation, and adapts. Until a controller runs usably, often millions of such attempts are needed. A real robot would need years for this and would likely break several times along the way.

In simulation, the same learning process takes hours to days. On powerful hardware, thousands of virtual robots can be trained simultaneously. Falls cost nothing, and spare parts are unlimited. Even dangerous situations can be practiced safely, such as an evasive maneuver by a self-driving car.

But the entire advantage evaporates if the controller fails on the real machine. This is exactly why sim-to-real transfer is a research question in its own right. The gap between simulation and reality even has its own name: the reality gap. It arises from friction, material wear, imprecise sensors, and delays in the electronics. All of this can only be calculated approximately.

How to close the reality gap

The most important technique is called domain randomization, meaning deliberate random variation. Instead of building a single, as-precise-as-possible simulation, one builds thousands of slightly different ones. Sometimes the floor is more slippery, sometimes the robot arm weighs a bit more, sometimes the motors respond more sluggishly. The controller learns across all these variants simultaneously.

The result is a controller that does not rely on exact values. It must be robust, because it never knows which variant it currently finds itself in. When it then encounters the real world, that is for it just another variant. One can compare it to a student who has practiced with a hundred different exercise problems instead of just one. In the exam, a new problem surprises them less.

A second approach works the other way around. One measures the behavior of the real robot and then adjusts the simulation until both match. This is called system identification. Often both are combined, and finally fine-tuned on the real machine at the end. A common misconception, by the way, is that better graphics help with this. For the physics, what matters is not how beautiful the simulation looks, but how realistically forces and contacts are calculated.

Legged robots, warehouses, and driving assistants

Sim-to-real transfer is most visible in four-legged walking robots. Models like Boston Dynamics' Spot or research robots from ETH Zurich move across gravel, stairs, and snow. Their gait control largely originated in simulation. There, it ran across millions of virtual surfaces before it ever touched real ground.

The technique has also arrived in industry. Gripper arms in warehouses learn in simulation to pick up unfamiliar packages. Car manufacturers train driving assistants in virtual cities, because rare accident situations can be generated there as often as desired. Nvidia sells a platform, Isaac Sim, precisely for this purpose, which regularly brings the term into business news.

In reports about humanoid robots, sim-to-real almost always comes up. When a company announces that its robot learned a task within a few days, simulation training is usually behind it. One should become skeptical when only videos from the simulation are shown. Only the performance in the real world proves that the transfer has succeeded.