System Card

When a company releases a new computer program that generates text or images, at first nobody outside the company knows how good it really is. A system card is the document meant to close this gap. In it, the manufacturer itself describes what the program is intended for and what it explicitly is not intended for. It also names typical errors, risky areas of use, and the tests it carried out before release. Such documents are usually a few dozen pages long and freely available online. You can think of them like the package insert of a medication: effects, side effects, warnings.

Why manufacturers write down their own weaknesses

AI systems are used today by millions of people, often for decisions with real consequences. A program that answers medical questions or pre-sorts job applications can cause harm if nobody knows its limits. Without public documentation, all that remains is advertising: manufacturers show the results that look good. A system card obliges them to also state the uncomfortable figures.

The second reason is regulation. With the AI Act, the EU has introduced documentation requirements for particularly influential systems. Anyone offering such systems must explain to authorities and users how they work and how they were tested. Many companies publish system cards voluntarily in order to get ahead of this obligation. It is also a matter of reputation: a provider without documentation appears untrustworthy to experts.

It is important to distinguish this from the older model card. A model card describes only the model, i.e. the trained core. A system card looks at the entire product around it: filters, blocks, additional tools, the user interface. This distinction is not a technicality, since many risks only arise from the interplay of the parts.

What such a document actually contains

At the beginning there are usually basic details: when was the system released, which languages does it understand, which types of data can it process. Then follow the test results. For this, the system is made to solve standardized sets of tasks, so-called benchmarks, and the success rates are given in percent. A value of 85 percent on math tasks sounds good, but also means: every seventh answer is wrong.

The most exciting part is the risk analysis. For this, companies hire experts who deliberately attack the system. This approach is called red teaming, named after the opposing team in military exercises. The testers try, for example, to get the system to reveal instructions for building weapons or to make racist statements. The system card then reports how often this succeeded and what was fixed afterward.

Finally, the protective measures are described. These include blocklists for sensitive topics, labeling of generated images, or usage caps. A common misconception is to take these documents for an independent review. They are self-disclosure. The manufacturer decides which tests to show and which to leave out.

System cards in product launches and market announcements

Practically every major model launch today is accompanied by a system card. OpenAI published such documents for GPT-4 and later versions, and Anthropic and Google proceed similarly. Anyone who reads them finds details missing from press releases: for instance, that a system performs significantly worse in certain languages.

For financial news, these documents have become an important source. Analysts read them to check whether a new model really has the advertised edge. Journalists also rely on them when reporting on risks. Some debates only arise because of a system card, since the manufacturer admits to a problem in it that nobody would otherwise have talked about.

For companies purchasing AI, the document is the basis for review. A health insurer or bank must be able to prove why it considers a particular system permissible. If a system card is missing, so is this proof. That is why the document is increasingly becoming a precondition for sale rather than merely a friendly add-on.