Redlines

Companies building programs with artificial intelligence define in advance what these programs are allowed to do. Much of this is a matter of weighing: benefit and risk are balanced against each other and decided case by case. Redlines are the part where no weighing takes place. They are boundaries that must never be crossed under any circumstances, no matter how useful or lucrative that might be in a given case. A typical example: a chat program must never provide a usable set of instructions for building a biological weapon. The term originates from diplomacy, where a red line marks the point beyond which a state will no longer cooperate.

Why absolute limits and not just weighing

Most rules for AI systems are formulated loosely. A model is supposed to be helpful, remain polite, and not assert false facts. Such rules can be bent in everyday practice, and that is often sensible. But for certain risks, this kind of weighing no longer works. If a harm cannot be undone, regretting it afterward does no good.

Redlines therefore create an area where there is no discussion at all. This has a practical side effect: developers are not under pressure to justify an exception in an individual case. The answer is already fixed before anyone even asks. This also guards against the gradual shifting of standards when a competitor happens to be moving faster.

For the public, redlines are also verifiable. A company that says “we act responsibly” is saying almost nothing. A company that publishes a concrete list can be measured against it. That is why regulators increasingly demand exactly such lists instead of general statements of intent.

From the list to an actual block

A redline initially exists only on paper. For it to take effect, it must be built into the technology at several points. This starts with training: problematic material is removed from the training data, and during training the model is systematically taught to refuse certain requests. A model that has never learned the answer also finds it harder to reveal it.

On top of that sits a second layer: small monitoring programs, so-called classifiers, that read along with every request and every response. They are trained to recognize critical topics and can cut off a response before it reaches the user. This layer sits outside the actual model and is easier to fix than the model itself.

Whether all of this holds up is tested through targeted attacks. A dedicated team, known in industry jargon as a red team, tries to get around the boundaries using tricks. Popular methods include fabricated role-play scenarios or breaking a forbidden question down into many harmless sub-questions. Whatever slips through gets patched. A common misconception is equating redlines with censorship: what is meant are very few, narrowly defined cases, not unwelcome opinions.

Redlines in company policies and in legislation

Major AI providers now publish their boundaries openly. Terms of use and safety reports list categories such as weapons of mass destruction, attacks on critical computer systems, or depictions of child abuse. Some providers tie these boundaries to risk levels: if a new model reaches a certain level of danger, it may only be released after additional safeguards are put in place.

Laws too operate on this logic. The European Union’s AI Act bans some applications entirely, including social scoring systems used by authorities to rate citizens. These are redlines backed by legal force rather than voluntary self-commitment. Violations can result in substantial fines.

In business news, the term usually surfaces when a boundary becomes contested. Typical cases involve military contracts or the question of whether a provider may freely offer its models for download. Anyone using an AI tool themselves notices redlines through a terse refusal without explanation. That harmless requests are occasionally blocked in the process is a cost that is knowingly accepted.