Instrumental Variable

Anyone who wants to know whether something has an effect is best off running an experiment. People are randomly split into two groups, only one of which is treated, and the two are compared at the end. Often, though, this isn’t possible: one may not, cannot, or does not want to randomly assign who studies, who smokes, or who gets a piece of software. In that case only real-world data remains, and there it is unclear what is cause and what is consequence. An instrumental variable is a way out of this situation. One looks for a circumstance that acts like a random draw, even though no one actually drew lots.

An example makes this tangible. Do people with more years of schooling earn more money? In the data, it looks that way. But perhaps more ambitious or wealthier people simply stay in school longer anyway. In that case, one isn’t measuring the effect of schooling but the effect of the parental home. Such hidden co-influences are called confounders.

Why correlation alone does not prove causation

In almost all real datasets, the variables of interest are related to one another. Countries with more doctors have more sick people. Students with expensive tutoring often have worse grades. Anyone reading the obvious cause into this has it exactly backwards. The statistics show a relationship, but not its direction.

In practice, this is costly. A company wants to know whether its advertising really drives sales. It places ads where purchasing power is high anyway. Afterward, the advertising looks fantastic, even though it may have accomplished nothing. The same applies to medications, social programs, and recommendation systems on the internet.

Instrumental variables are one of the few methods that address this problem cleanly without requiring an actual experiment. They originated in economics but are now also used in medicine, epidemiology, and machine learning. The 2021 Nobel Prize in Economics went, among others, to researchers who decisively shaped this technique.

The three conditions for a good instrument

An instrument is a third variable. It must satisfy three things. First, it must noticeably influence the suspected cause. Second, it may only influence the outcome through this detour, not directly. Third, it must not itself depend on the hidden confounders.

Back to years of schooling. A famous instrument is month of birth. In many countries, compulsory schooling applies up to a certain age. Someone born early in the year may therefore leave school somewhat earlier and accumulates, on average, fewer years of schooling. Month of birth thus affects years of schooling, but is neither ambitious nor wealthy. It is essentially randomly distributed.

Computationally, one proceeds in two steps. First, one estimates how much of years of schooling can be explained by month of birth alone. Then one checks how strongly this explained portion is related to later income. Only the portion of schooling triggered by the instrument enters the calculation. The remaining, possibly contaminated portion is left out.

The catch lies in condition two. It cannot be proven from the data, only justified substantively. Is month of birth really harmless, or is it perhaps related to health? Another risk is weak instruments: if the connection to the cause is only tiny, the result becomes extremely imprecise.

Instruments in economic news and in AI systems

In economic news reports, instrumental variables often lurk behind sentences like “a study causally shows that ...”. Typical instruments are weather events, changes in law on a specific cutoff date, distances to the nearest clinic, or lotteries in the allocation of study places. What they all have in common is that they sort people into groups without any action on their part.

Technology companies use the idea too. When a platform introduces a new feature, not everyone uses it. Instead of simply comparing users of the feature with everyone else, the random assignment to the feature is used as an instrument. This is essentially a variant of A/B testing, i.e., the controlled comparison of two versions.

In machine learning, the field of causal models is growing. A predictive model only recognizes patterns in data. It cannot say what happens if one actively intervenes. Exactly for this purpose, neural networks are combined with instrumental variables, for instance in pricing or in drug research. Anyone reading such reports should always ask one question: which instrument was used, and is it credible?