End-to-End Test

Software almost always consists of many individual parts. In an online shop, for example, these are the website, the search function, the database with the products, and the payment service. Each of these parts can be tested on its own. An end-to-end test does something different: it runs once through the entire chain, from one end to the other. An automated program clicks through the site like a customer, searches for a product, adds it to the cart, and pays. At the end, it checks whether the order confirmation appears and whether the order was actually saved.

Errors that only appear in the interaction between parts

The most dangerous errors rarely arise in a single component. They arise at the transitions between them. One example: the payment service expects the amount in cents, but the website sends it in euros. Both parts work perfectly correctly on their own. Yet the customer suddenly pays a hundred times too much.

Such errors are only visible when you run through the complete process. This is exactly where the value of these tests lies. They are the last check before a change goes out to real users. Major providers release new versions of their software several times a day. Without automated runs, this pace would not be responsible.

The price for this is high. End-to-end tests are slow because they start real systems and wait for responses. They also sometimes fail even though there is no real error—for instance, because a page takes a second longer to load. Developers call such unreliable tests “flaky.” That’s why teams build few end-to-end tests for the most important processes and many fast tests for the individual parts.

How a test run works

An end-to-end test is written as a program, usually just a few dozen lines long. It contains a sequence of instructions: open this address, type this text into this field, click this button. These are followed by checks, called assertions. They specify what must be visible afterward, for example the sentence “Thank you for your order.”

This is executed by tools such as Playwright, Cypress, or Selenium. They control a real browser, just without a window on the screen. So the browser does exactly what a human would do, only faster and always identically. If it fails, the tool saves a screenshot and a video of the run. This way you can see exactly where things went wrong.

So that the tests don’t shop in real systems, they run in a test environment. This is a copy of the application with made-up data and a payment service in practice mode. The runs are triggered automatically as soon as someone submits new program code. If a test fails, the change is not released.

E2E tests in AI products and in company reports

As a user, you only notice these tests indirectly—namely when they were missing. A banking app in which the transfer fails after an update is a typical case. Reports about such outages often mention “gaps in test coverage.” This usually refers to exactly this missing run through the entire chain.

With AI products, the matter is particularly tricky. A language model doesn’t always respond to the same question with the same words. A test therefore cannot check for an exact sentence. Instead, coarser properties are checked: Does a response come at all? Does it contain the number being searched for? Does it stay within the allowed waiting time? Some teams have a second model evaluate the response.

The end-to-end test should not be confused with the unit test. A unit test checks a tiny piece of code in isolation and runs in milliseconds. The end-to-end test checks the whole thing and takes seconds to minutes. Both belong together: one tells you which part is broken, the other tells you whether the product still works for the user.