Backdoor

A backdoor is a hidden access point to a system that circumvents normal security checks. Someone who knows about this backdoor can control the system without having to log in or authenticate through the regular process. Backdoors exist in classical software, in operating systems, and increasingly also in AI models. They can be deliberately built in — for example by an attacker — or arise unintentionally. The crucial point is: from the outside, the system behaves normally. Only someone who knows about the hidden access can exploit it.

Backdoors in AI models

In classical IT, a backdoor is usually a vulnerability in software or a secretly built-in password. With AI models, things are more sophisticated. Here, an attacker can manipulate the model during training — that is, during the phase in which it learns from data — so that it reacts incorrectly to a specific secret signal.

This signal is called a trigger. It can be a specific word, a pattern in an image, or even a barely visible watermark. As long as no trigger appears, the model behaves completely inconspicuously. As soon as the trigger appears, it delivers a predetermined, incorrect output — for example, it misclassifies an image or issues a dangerous instruction.

This is particularly dangerous because standard tests do not uncover this behavior. A model can pass all benchmarks — these are standardized performance tests — and still contain a backdoor. This is why it is also referred to as a sleeper attack: the vulnerability is present, but only reveals itself at the right moment.

How an AI backdoor arises

The most common route is through the training data. Anyone training a model needs huge amounts of examples — this data is often collected from the internet or obtained from third parties. An attacker can deliberately smuggle manipulated examples into this data. The model then unknowingly learns to treat the trigger as a signal. This attack is called data poisoning.

A second possibility is direct intervention in the model itself. Anyone with access to a model’s weights — these are the countless internal settings that the model adjusts during training — can alter them in a way that creates a backdoor. This is particularly a risk when companies download ready-made models from the internet instead of training them themselves.

Carefully checking training data and sourcing it from trustworthy origins helps guard against data poisoning. Cryptographic verification protects against manipulated model weights — this means using mathematics to verify that the model is unaltered and originates from the original provider.

Where backdoors show up in practice

In classical cybersecurity, backdoors have been known for decades. Government intelligence agencies occasionally demand that technology companies build such access points into encrypted communications — on the grounds that this allows better surveillance of criminals. Security experts have long warned that a backdoor for authorities is also a backdoor for attackers.

In the AI field, the topic is younger but is rapidly gaining importance. Platforms like Hugging Face, where hundreds of thousands of models are publicly shared, are a potential entry point. Anyone using someone else’s model for their own application cannot easily be certain that it is unmanipulated. Researchers have shown that backdoors are possible in image recognition systems, text systems, and even in medical AI applications.

The topic is also relevant for companies that purchase AI models rather than developing them themselves. The more AI is deployed in critical areas such as financial or infrastructure systems, the higher the risk becomes — and the more important it becomes not to blindly trust models.