Quality Gate

A quality gate is a checkpoint within a workflow. At this point, it is checked whether a result meets previously defined conditions. Only if the check is passed does the work proceed. Otherwise, the process is halted and the result must be reworked. The comparison to a barrier at a railway crossing fits well: it is either open or closed, there is no such thing as letting a little through. What matters is that the conditions are agreed upon in writing beforehand. Otherwise, it is the mood of those involved that decides at the barrier.

Why nobody voluntarily installs a barrier

Errors become more expensive the later they are found. A typo in program code is fixed in five minutes. The same error in software running for a million customers costs recalls, support, and reputation. A quality gate is an attempt to catch the error as early as possible, that is, while it is still cheap.

The second reason is commitment. Every project faces deadline pressure. Without a fixed checkpoint, the check is skipped exactly when it would matter most, namely shortly before the deadline. A quality gate takes this decision out of the moment. The rule is already established before the pressure arrives.

With AI systems, a third reason is added. A language model, that is, a program that generates text, can suddenly produce worse or more sensitive answers after a change. This is not noticed by simply looking, but only through systematic testing. Exactly such tests are now often built in as a quality gate before a release.

What is measured at the barrier

A quality gate always consists of measurable criteria and a threshold. One criterion, for example, is: at least 80 percent of the program code must be covered by automated tests. Or: it must not contain any known severe security vulnerability. What matters is that this can be checked without discussion. Statements like “the code should be nice” do not qualify as a criterion.

In software development, the check usually runs automatically. As soon as someone submits a change, a server starts the tests and calculates the metrics. If a metric falls below the threshold, the change is rejected. The human doesn’t need to do anything for this, and that is exactly the point: an automatic barrier cannot be bypassed out of convenience.

With AI models, the criteria are harder to define. The model is then made to answer a fixed set of test questions, and the results are compared with the previous version. Typical thresholds are: the hit rate must not decrease, and the number of undesired responses must stay below a limit. A common misconception is to confuse a quality gate with a guarantee. It only checks what is stated in the criteria. Anything nobody thought of passes through unhindered.

From the car factory to the AI release

The term originates from industry, especially from automobile manufacturing. There, fixed points exist in the development of a new model at which the release for series production is checked. If a quality gate is not passed, the start of production is postponed. The software industry adopted the principle from this way of thinking.

Today, quality gates are encountered mainly in tools for developer teams. Anyone working on platforms like GitHub knows the red and green checkmarks on a change. Behind them lies exactly this principle. Code analysis tools also use the term literally as a label for their rule sets.

In corporate news, the term usually appears in connection with regulation and liability. When a company has to explain how it tested an AI before deployment, it likes to point to its quality gates. Related, but not the same, is the term guardrail: this refers to protective mechanisms that intervene during ongoing operation. A quality gate acts beforehand, a guardrail acts during.