Held-out split

Anyone who lets a program learn from examples faces a problem: how do you tell whether it has actually understood something? If you test it on exactly the examples it practiced with, the result tells you little. That is why the data is split from the very beginning. One part is used for practicing, another part is set aside and never touched during practice. This set-aside part is called the held-out split. Only at the end is it brought out to see how well the program handles it. It is the same idea as with a class test: it contains tasks the student has not seen before.

What withholding data reveals about real ability

A learning program can perform well in two very different ways. Either it has recognized general patterns, or it has simply memorized the practice examples. From the outside, both look the same as long as you only test it on the practice examples. The held-out split separates the two cases. Whoever has understood the patterns will also do well on new examples. Whoever has merely memorized will clearly fall short here.

There is a technical term for this kind of memorizing: overfitting. The program adapts so precisely to its training data that it also picks up on its random quirks. Without withheld data, such an error often goes unnoticed until the program fails in real-world use. This is exactly why the held-out split is not an optional extra in practice, but a basic requirement.

There is a second reason, and it is economic. Companies compare models using percentage figures and base decisions worth millions on them. Such figures are only reliable if they were measured on unseen data. A success rate on training data is, in doubt, inflated.

How the data split works

Usually the data is split into three parts. The largest part, often around 80 percent, is used for practicing. A small part is used for fine-tuning, for example to decide how long to practice. The last part, often 10 percent, remains untouched until the very end and is the actual held-out split. The names for these are training data, validation data, and test data.

It is crucial that the split happens randomly and fairly. Both parts should contain the same kind of examples. If, by mistake, only easy cases end up in the withheld part, the result looks better than it actually is. For data with a time component, such as stock prices, the split is instead made by date: practice is done on the past, testing on the future.

The most common mistake is called data leakage. In this case, information from the withheld part slips into the practice data after all. This happens quickly, for example when the same photo, slightly cropped differently, ends up in both parts. After that, the held-out split is worthless, because the program has already seen the test questions. Those who have very little data instead use cross-validation: the split is repeated several times, withholding a different section each time.

Held-out data in model comparisons and leaderboards

You encounter this term everywhere AI systems are compared with one another. Well-known benchmark collections for language models sometimes do not release their tasks publicly at all, but keep them withheld. This is the only way to prevent the tasks from simply being learned in the next training run. When a company reports a top score on such a test, this is the reason the number carries any weight at all.

In companies, the held-out split is part of quality control. A bank that wants to predict loan defaults tests its model on withheld historical cases. An online shop tests its recommendations on purchases the model has never seen. Regulators, too, are increasingly asking on what data the stated accuracies were measured.

In the news, withheld data often appears in the form of a suspicion. It is repeatedly claimed that a model had already seen the test tasks during training. The accusation is then called contamination, meaning the test data has been tainted. Anyone who understands the held-out split can correctly assess such reports: without clean, unseen test data, any top score is merely a claim.