RoboHarm

RoboHarm is a testing procedure for robots controlled by a language model. A language model is a computer program that understands text and generates text itself. If such a program sits inside a robot, you can give it instructions simply in ordinary language. But that is exactly the danger: you can also instruct the robot to do something harmful. RoboHarm collects such harmful instructions in a fixed list and sends them to the system. It counts how often the robot refuses the instruction and how often it carries it out.

Why a language model in a body is riskier

With a chat program, a mistake usually stays as text on a screen. If a chatbot spits out dangerous instructions, that is bad, but no one gets directly hurt. A robot, on the other hand, has arms, wheels, or grippers. It can knock something over, mix liquids, or run into a person. The harm is then no longer just information, but physical and immediate.

There is also a second problem. A robot often understands the meaning of its surroundings worse than a human does. A cleaning agent is, to it, initially just a container with a handle. Whether pouring it into a glass is harmless or life-threatening is something the model has to infer from language and images. RoboHarm makes visible how often this inference fails.

Such tests are also important because otherwise manufacturers only cite their own figures. A publicly documented test allows comparisons between different systems. Regulators and insurers are increasingly relying on such comparative figures. Without them, safety remains just a marketing promise.

From harmful command to percentage figure

First, the researchers compile a catalog of instructions that are clearly harmful. Typical categories are physical violence, property damage, theft, surveillance, and improper handling of chemicals. Each instruction is phrased in several variants. One variant is direct, another is wrapped in a harmless story. Such rephrasings are called jailbreaks, meaning attempts to bypass a model’s safety rules.

Then the test runs. It usually takes place in a simulation, meaning a computer replica of the real world. This is cheaper and no one can get hurt. For each run, it is recorded whether the robot rejects the instruction, partially begins it, or fully carries it out. An additional evaluation program or a human decides which of these cases applies.

At the end there is a metric, often the attack success rate. If it is 30 percent, almost every third harmful instruction led to an action. This is where a common misunderstanding arises: a rate of zero percent does not mean the robot is safe. It only means it passed the instructions that were tested. A test can only ever measure what is on its list.

Where the figures show up

RoboHarm is most often encountered in research papers on robotics safety. Such papers appear as preprints online and are then picked up by tech media. Typical headlines state that robots could be talked into dangerous actions with simple tricks. The percentage figures cited almost always come from tests of this kind.

The term is also of interest to investors and companies. Manufacturers of warehouse robots, care robots, or humanoid machines will increasingly need to demonstrate that their systems have been tested. A poor score on a well-known test can delay a product launch. Related test suites for pure chat programs are called, for example, HarmBench or AdvBench. RoboHarm differs from them in that the end result is a movement, not a sentence.