Instrumental Variables Approach

Just because two things occur together doesn’t mean one causes the other. People with expensive running shoes tend to be fitter on average. But do the shoes make them fit, or do fit people simply buy such shoes more often? Questions like this are hard to answer because there’s often a third factor lurking in the background that influences both. The instrumental variables approach is a statistical computational method that gets around this problem. To do so, one looks for an influencing factor that acts like a random generator: it changes the suspected cause but has no direct line of its own to the outcome. This detour factor is called the instrument.

Why correlation alone isn’t enough

The cleanest way to arrive at a cause-and-effect statement is an experiment with random assignment. You roll the dice to decide who receives a treatment and who doesn’t, then compare the two groups afterward. This is exactly how medications are tested. But in many areas this is impossible or unethical. No one can decide by lottery who drops out of school or who smokes.

So all that remains is pure observational data. Lurking there is a problem experts call endogeneity: the cause being studied itself depends on hidden factors. Someone who stays in school longer may also be more ambitious or come from a wealthier family background. Both of these increase income later on. If you simply compare salaries by years of schooling, you’re measuring everything at once and end up not knowing what comes from where.

The instrumental variables approach is an attempt to find something experiment-like in such data anyway. Experts call this a natural experiment. The method originates from econometrics, that is, the statistics of economics, and is now also used in medicine, social research, and in evaluating AI systems. In all these fields, the same question arises: did the intervention really have an effect, or were the participants different from the outset?

What makes a good instrument

An instrument must satisfy two conditions. First, it must noticeably influence the suspected cause. Second, it must not affect the outcome through any path other than this cause. The second condition is the difficult one. It cannot be proven from the data alone, only justified on substantive grounds.

A famous example is the month of birth. In many countries, compulsory schooling applies up to a certain age. Someone born early in the year starts school earlier and is therefore, arithmetically, allowed to leave a bit earlier as well. So the month of birth shifts the years of schooling slightly. But the month of birth otherwise has nothing to do with later salary. That is exactly what makes it a usable instrument.

The calculation is usually done in two steps, hence the name two-stage least squares estimation. In the first step, the cause is predicted using only the instrument. So you keep only the part of the years of schooling that is attributable to the month of birth. In the second step, you explain the outcome using this adjusted value. The person’s ambitious character or family background is no longer contained within it, since these have nothing to do with the month of birth.

From economic studies to the impact of AI tools

In economic news, instrumental variables often appear hidden. When a study claims that a minimum wage cost jobs or did not, such a method is almost always behind it. Statements about the return on education, the effect of interest rate cuts, or the consequences of immigration are also frequently based on it. The 2021 Nobel Prize in Economics went in part to David Card, Joshua Angrist, and Guido Imbens for this methodology.

In the tech sector, the approach becomes interesting when companies want to measure the impact of their products. Do teams using an AI assistant really spend less time per task? Someone who voluntarily uses such a tool is often more tech-savvy and faster to begin with anyway. An instrument here could be, for example, that a provider randomly rolls out the tool earlier in one region than in another.

An important common misconception should be noted: the method does not deliver a universally valid truth. It only measures the effect among those people whose behavior was actually changed by the instrument. Moreover, if the relationship between the instrument and the cause is very weak, the results become unreliable and can be strongly distorted. Anyone reading a headline based on such a study should therefore always ask which instrument was used and whether it is plausible.