Self-replicating

Something is self-replicating if it produces copies of itself without a human triggering every step. The model for this is nature: a bacterium divides, one becomes two, two become four. No external blueprint is needed, because the instructions are contained within the thing itself. In technology, this idea is applied to programs. A self-replicating program thus contains not only its task but also the instruction to propagate itself. The term is neutral: it describes how something spreads, not whether it causes harm.

Why spread without a brake is so hard to control

Self-replication is a security concern precisely because it grows exponentially. If a program copies itself once per hour, after ten hours there are already over a thousand copies. Such growth outpaces any manual response. This is exactly what distinguishes a self-replicating program from normal software, which only runs where someone has installed it.

A historical example is the Morris Worm of 1988. A student released it to map the internet. Due to a bug, it copied itself far too often and crippled thousands of computers. The author did not intend to cause harm. Nevertheless, the damage was extensive, because a self-replicating system cannot simply be recalled.

In the AI debate, self-replication is considered a so-called dangerous capability. Major AI labs therefore test their models for this before release. The test question is: could the model independently rent a server, install itself there, and keep running? So far, models fail at such tasks because they cannot reliably sustain long chains of action without errors.

What a program needs to copy itself

Technically, self-replication requires three things. First, its own code as data, so the program actually knows what it is supposed to copy. Second, a way out, such as a network, a USB drive, or an email. Third, a trigger that starts the copy at its destination.

The third point is the most difficult. A file on a foreign computer does nothing by itself. Classic worms exploit security vulnerabilities for this: flaws in software that allow foreign code to be executed without permission. Other malware relies on humans, luring them with a seemingly harmless attachment. Both achieve the same goal, namely starting the copy.

A common misconception is that every computer virus is self-replicating. A virus attaches itself to another file and requires that file to be executed, whereas a worm spreads independently over networks. A trojan, by contrast, merely disguises itself and usually does not copy itself at all. Likewise, AI systems that write their own program code are not automatically self-replicating. The term only applies once the result recreates and launches the system itself.

From antivirus software to safety tests at AI labs

In everyday life, this topic is encountered mainly as defense. Virus scanners, automatic updates, and firewalls exist largely to break copying chains. The fact that email programs block executable attachments is also part of this. These measures act precisely at the point where a copy would need to be launched.

In business news, the term appears when a major malware wave hits companies. Well-known cases such as WannaCry or NotPetya paralyzed hospitals, shipping companies, and factories in 2017. The damages ran into the billions because the programs spread on their own within corporate networks. For insurers and stock markets, such events have since become a distinct risk category.

In AI news, self-replication appears in the safety reports for new models. These reports state whether a model shows signs of this capability. If the test comes back negative, this is taken as an argument for release. Researchers also discuss the topic in nanotechnology and spaceflight, for instance regarding robots meant to self-replicate on the Moon. There, self-replication would be explicitly desired.