Observation Data

Observation data is data you simply collect by watching. Nobody alters the situation in the process; you just record what would happen anyway. One example: an online shop stores which products customers click on and buy. The shop didn’t prescribe anything to the customers, it just wrote down what happened. The counterpart to this is an experiment: there, you deliberately intervene and see what changes as a result. The German term is 'Beobachtungsdaten,' but in professional circles you almost always read the English form.

Why observation is not the same as explanation

Observation data is cheap and available in huge quantities. Every app, every checkout, and every sensor produces it as a byproduct. This is exactly what most AI systems in use today are built on. A language model was trained on texts that people wrote for entirely different reasons. Nobody planned these texts for training.

The big catch is the question of cause and effect. Observation data shows what occurs together, not what causes what. A well-known example: in months with high ice cream sales, there are more drowning accidents. But the ice cream doesn’t cause the accidents. Both are linked to hot weather, a hidden third cause.

For companies, this distinction is costly. A model might see that customers with a newsletter subscription buy more. Whether the newsletter actually causes this, it doesn’t know. Perhaps only the most loyal customers subscribe to it anyway. Anyone who shifts their marketing budget based on this can burn through a lot of money.

How recorded traces become a dataset

It starts with what’s called a log, an automatically maintained record file. Servers, apps, or machines continuously write events into it with timestamps. This raw data is messy: entries are missing, duplicated, or exist in different formats. A large part of the work consists of cleaning it up and bringing it into a uniform table.

After that, experts try to keep an eye on the biases involved. Observation data only shows what was actually observed in the first place. A bank has repayment data exclusively from people it granted a loan to. It knows nothing about those who were rejected. If you train a model blindly on such data, it learns the old decisions, not reality.

There are statistical methods that estimate causes even without an experiment. This involves comparing groups that are as similar as possible and calculating out known confounding factors. This is called causal inference. The result is never certain, since an unknown confounding factor can always remain. That’s why a controlled experiment is still considered the more reliable approach.

From click paths to studies in medicine

In everyday life, you constantly generate observation data. Streaming services remember which episodes you watch to the end. Navigation apps collect speed data from thousands of phones and use it to detect traffic jams. Fitness watches log your pulse and step count over months. In all these cases, nobody intervenes in your behavior.

In the news, the term often comes up in the context of medicine and regulation. So-called real-world data from clinics and health insurers is observation data. It shows how medications perform outside of clean studies. Regulatory bodies increasingly accept it as a supplement, but rarely as sole proof. The reason is the same again: whoever receives a medication is usually not chosen at random.

A common misconception is the assumption that a very large amount of data would solve the problem. This isn’t true. A bias isn’t reduced by sheer volume — it’s only measured more precisely. That’s why large tech companies additionally run real experiments, usually called A/B tests. In these, two randomly formed groups of users see different variants of a page. Only this randomness makes the comparison meaningful.