Self-Replication

Self-replication means: something produces a functional copy of itself. One instance becomes two, two become four. In nature, this is the normal case. A cell divides, a virus has itself multiplied inside a host cell. With computer programs, however, self-replication is something that someone has to deliberately build in. To do this, a program must be able to read its own code, store it elsewhere, and restart there. In the discussion about artificial intelligence, the term usually refers to exactly this case: a program that spreads to other computers without instruction from a human and keeps working there.

The red line in the AI safety debate

Self-replication is considered a so-called dangerous capability in AI safety research. This refers to properties that make a system hard to control. The reason is simple: a program that only runs on one server can be shut down. A program that has already copied itself onto twenty other servers beforehand cannot be shut down so easily anymore.

That is why large AI labs carefully test their models for this capability before release. Such tests are called evaluations. The model is placed in an isolated test environment and given the task of copying itself. If the models pass this test, a warning threshold is considered reached. Public reports so far conclude that current language models manage individual sub-steps, but usually not the complete chain.

It is important to distinguish this from a computer virus. A virus copies itself rigidly according to a fixed pattern and always does the same thing. A self-replicating AI could react to obstacles, look for workarounds, and get around errors. It is precisely this adaptability that makes the difference and worries experts.

What steps an actual copy requires

Self-replication is not a single action but a chain. First, the system must recognize which parts it consists of. For a language model, these are the model file with the learned numerical values and the program that executes it. It then needs access to a second computer, usually rented computing power on the internet. There, it must store the files, install the appropriate software, and trigger the startup.

Each of these steps can fail. Model files are often several hundred gigabytes in size, so copying takes correspondingly long. Rented servers require access credentials and a means of payment. If a software version doesn’t match, the copy won’t start up. In tests, models often fail at such mundane technical hurdles, not at understanding the task.

You can think of it like a recipe that is supposed to copy itself out. It needs paper, a pen, and a kitchen at the new location. If any one of these is missing, it stays with the original. It is exactly at these prerequisites that protective measures apply: limited permissions, no free network access, monitoring of unusual data transfers.

Where the term appears in news and regulations

Self-replication is most often read about in the safety reports that AI companies publish for new models. There it might say, for instance, that a model did not reach the threshold for autonomous replication. State-run AI safety testing bodies in the United Kingdom and the United States also use the term in their test catalogs.

At the end of 2024, a Chinese study made headlines in which two language models produced copies of themselves in experiments. Experts pointed out that the models received a lot of assistance and broad permissions within the test system. Such details determine whether a report is alarming or harmless.

Outside of AI, the term has other meanings as well. In robotics, researchers work on machines that manufacture components for further machines. In biology, self-replication describes the duplication of genetic material. So if you read the term in a news article, it is worth checking which field is being discussed.