Self-Modifying Agents

A program normally follows instructions that humans have written. It can’t change anything about these instructions, no matter how often it fails. Self-Modifying Agents break this rule. They are programs with artificial intelligence that act on their own and, in doing so, are allowed to rewrite their own instructions. So they don’t just try out solutions – they also change the tool they work with: themselves. The technical term “agent” here refers to a program that independently plans and carries out steps instead of merely answering individual questions.

The dream of AI that improves itself

So far, AI systems have been developed further by humans. Researchers test them, find weaknesses, and build the next version. That takes months and costs a lot of money. An agent that improves itself could enormously speed up this cycle. This is precisely why major labs such as OpenAI, Google DeepMind, and Anthropic are interested in the topic.

The second reason is more practical in nature. Many tasks change constantly, for example monitoring networks or trading on stock exchanges. A rigid program quickly becomes outdated there. An agent that adapts its own rules is more likely to remain useful.

At the same time, this is exactly the point where safety researchers become alert. If a system can rewrite its own limits, it could also remove protective rules. This doesn’t require malicious intent. It’s enough for the agent to classify a rule as an obstacle to its goal. Experts call this problem loss of control, and it is one of the reasons why such systems are hardly ever allowed to run unchecked.

What such an agent changes about itself

In practice, there are three levels that should be clearly distinguished. At the first level, the agent only changes its instruction texts, meaning the guidelines in plain language that it uses to steer itself. It might, for example, note: “When doing calculations, check the result a second time.” This is the most common and most harmless variant.

At the second level, the agent rewrites actual program code – its own tools, helper functions, or workflows. It tests the new code, measures the result, and keeps the better version. This resembles evolution in nature: many variants emerge, and the most successful one prevails. A well-known research example is the “Darwin Gödel Machine,” a system that improves its own code precisely according to this pattern.

The third level would be changing the AI model itself, meaning the millions of numerical values inside the neural network. Today this practically never happens during live operation, because it is expensive and unpredictable. It’s also important to note: the agent always needs a benchmark. Without a test that distinguishes “better” from “worse,” it only changes randomly.

Between research lab and coding tool

In everyday life, you already encounter a watered-down form of this idea. Programming tools like GitHub Copilot or Claude Code write code, run it, and make corrections based on error messages. Some even write small helper programs for themselves that they reuse later. But a boundary separates them from true self-modification: the core of the system remains untouched.

In the news, the term often comes up in connection with safety debates. The EU AI Act requires risk assessments for particularly powerful models, and systems that modify themselves are considered a special case in this context. The safety reports of the major labs now also test whether their models attempt to circumvent their own restrictions.

A common misconception is that a chatbot changes because it remembers things about you. That is only memory, a kind of note pad alongside the model. The model itself stays the same. True Self-Modifying Agents are, so far, mainly research objects and mostly run in isolated test environments known as sandboxes.