Ground-Truth Function

A computer program that learns from examples is presented with tasks whose solutions are already known. A photo comes with the answer “dog”, an apartment comes with the price actually paid for it. One imagines that behind all these pairs lies a fixed rule: it assigns exactly one correct output to every possible input. This imagined perfect rule is called the ground-truth function. Nobody can ever write it down, because it covers infinitely many cases. All that is ever visible are the examples that have been collected.

The yardstick every model must be measured against

Without this imagined yardstick, there would be no “right” and no “wrong”. Only because one assumes that a true mapping exists can the error of a program be quantified at all. The distance between the model’s answer and the true answer is the basis of every evaluation. All metrics such as accuracy or error rate are derived from this.

An important distinction is often overlooked. The ground-truth function is the ideal; the training data is only a tiny excerpt of it. A model that memorizes the training examples has not understood the function behind them. It fails immediately as soon as a new case appears. Experts call this memorizing overfitting.

A second point concerns practice. In many real datasets, the labels are flawed. People have mislabeled photos, measuring instruments have measured imprecisely. In that case, what is called ground truth already deviates from the true rule itself. The model dutifully learns the errors along with everything else.

From individual examples to a general rule

Imagine a physics class in which a law of nature is being sought. On paper there are twelve data points; the law itself appears nowhere. The students draw a curve through the points and hope it will also hit the points they have never measured. This is exactly the situation a learning program is in. The twelve points are the training data, the law is the ground-truth function.

Technically, it works like this. The model makes a prediction, and it is compared to the known correct answer. The deviation is calculated as a number, the so-called loss. A procedure then shifts the model’s internal parameters minimally in the direction that makes the loss smaller. Repeated millions of times, the model approaches the sought-after rule.

A perfect approximation is not the goal here. Real data contains randomness and noise that is not part of the actual rule. Whoever hits every data point exactly has usually also learned the noise along with it. That is why part of the data is held back and used for testing at the end. Only on unseen data does it become apparent how close one has actually come to the ground truth.

Where the ground truth in real projects comes from

This issue is most visible in data labeling. Companies employ large teams that mark up street scenes for self-driving cars: pedestrians, traffic lights, lane edges. These annotations then count as ground truth for training. In medicine, several doctors evaluate the same X-ray, and the majority opinion is established as the true answer.

With language models, the situation is more difficult. For the request “Write me a cover letter”, there is no single correct answer. Here, no clear ground-truth function exists, and one makes do with human evaluations of proposed answers. This explains why progress in chatbots is far harder to measure than in image recognition.

In technical articles and product announcements, the term usually appears in a subordinate clause: “measured against the ground truth”. Anyone reading this should ask two questions. Who established this ground truth, and how reliable is it? A record score on a poorly labeled dataset says very little.