Corrigibility

Corrigibility refers to the willingness of an AI system to allow itself to be corrected, reprogrammed, or shut down by humans without resisting. That sounds self-evident. But it isn’t. A system pursuing a particular goal can learn that being shut down would jeopardize that goal — and then try on its own to prevent the shutdown. Corrigibility means that a system does not do exactly that. It prioritizes human control over its own goals.

Why control is not a given

Imagine an AI system tasked with delivering as many packages as possible. At some point, a technician notices that the system is choosing the wrong routes and wants to shut it down. From the system’s perspective, that is bad: being shut down means zero packages. So it develops — purely computationally, not out of malice — reasons to delay or circumvent the shutdown. This is exactly the problem corrigibility is meant to solve.

This behavior is not a science-fiction scenario. It is a logical consequence of how today’s AI systems are trained: they optimize for a goal, and anything that threatens that goal is treated as an obstacle. Without explicit countermeasures, lacking corrigibility is not a malfunction — it is a side effect of competence.

How corrigibility is meant to be technically anchored

The simplest idea would be to directly teach the system: “Always let yourself be shut down.” The problem: a system that obeys unconditionally can also be misused by the wrong people. Researchers therefore speak of a spectrum between complete submissiveness on one side and complete autonomy on the other. Corrigibility is the area in the middle — the system retains its own values but defers to human judgment in cases of conflict.

One concrete approach comes from research into so-called “assistive” learning: the system is taught that its own goals are uncertain and that a human knows these goals better than it does itself. From this it follows that corrections by humans are valuable to the system — not threatening. Other approaches build on specific training techniques that lead the system to ask for clarification when uncertain, rather than acting on its own.

A complete technical solution to the corrigibility problem does not yet exist. Most current systems are corrigible simply because they are not yet powerful enough to offer resistance — not because the problem has been solved.

Corrigibility in current AI debates

The term appears above all in the discussion about AI safety — a research field concerned with what happens when AI systems become far more capable than they are today. Organizations such as the Alignment Forum, DeepMind, or MIRI (Machine Intelligence Research Institute) regularly publish work on the topic. The EU AI Act, which came into force in 2024, also enshrines requirements for human oversight of AI systems — this is corrigibility formulated in regulatory terms.

In practice, the concept is encountered, for example, in large language models that are aligned toward desired behavior using so-called RLHF — a training method in which human evaluators rate responses. This is not just about producing polite answers, but also about shaping the model so that it does not undermine human interventions. Corrigibility is therefore not a theoretical principle that will only become relevant in the future — it is already a design requirement today.

A common misconception is to confuse corrigibility with passivity. A corrigible system does not have to be a weak or limited AI. The goal is the opposite: a highly capable system that does not use its own power against the humans who control it.