Quality Assurance (QA)

QA is the abbreviation for the English term “Quality Assurance”. It refers to everything a company does to find flaws in its product before customers find them. In the automotive industry, brakes and welds are checked for this purpose; in the software industry, programs are checked. There, people and automated testing tools systematically try out whether an application does what it is supposed to do. It’s important to distinguish this from mere testing: testing is a single check, while QA is the entire organized process surrounding it. This includes rules, checklists, responsibilities, and the question of when something may even be released.

What an overlooked error costs

Errors become more expensive the later they are discovered. A programmer who spots their mistake after ten minutes fixes a single line. If the same error only becomes visible after release, support, development, and management all have to get involved. At banks or hospitals, a software error can also cause real harm, not just annoyance.

That’s why QA is also a financial matter for publicly traded technology companies. A faulty update can take down an online shop for hours and cost millions in revenue. Recalls of cars with software defects hit profits directly. Investors therefore pay attention to how reliably a company delivers its products.

With AI systems, there’s an additional reason: a language model can invent false claims without anything crashing. So there’s no error message that would reveal this. Only targeted testing uncovers such problems.

From unit tests to release

Classic software QA works in stages. First, so-called unit tests check individual small building blocks of the program, such as a function that converts a date. After that, one tests whether several building blocks work together. Finally, someone clicks through the finished application like a real user. Most of these checks run automatically, every time a developer submits new code.

You can picture this like the inspections in a large bakery. First every ingredient is checked, then the dough, then the baked bread. If something is noticed right at the end, you at least roughly know where it went wrong.

With AI models, this works differently, because there is often no single correct answer. Here, one works with benchmarks, meaning standardized collections of tasks where the hit rate is counted. In addition, there are reviewers who evaluate answers by hand, and so-called red teaming: experts deliberately try to lure the model into dangerous or false statements. A typical mistake is confusing good benchmark scores with good real-world quality — models can become downright overfit to well-known test tasks.

QA in job postings and update notices

The term appears everywhere in the working world. “QA Engineer” is its own profession, designing and automating testing procedures. Game companies have QA departments that play through new titles for weeks to log crashes. If you read in software news that a release is “in QA,” you now know: it’s finished being programmed, but not yet released.

Users also notice QA indirectly. Beta versions of apps are one form of this, since volunteers report bugs. If a company pulls back an update on short notice, QA missed something beforehand. Beware of the abbreviation: in AI research, “QA” sometimes also stands for “Question Answering,” meaning systems that answer questions. What is meant usually becomes clear from context.