Cobra Effect

The name goes back to a story from the British colonial era in India. The administration wanted to reduce the number of venomous snakes in Delhi. So it paid a bounty for every dead cobra handed in. Enterprising people then began breeding cobras themselves in order to trade them in for money. When the authorities noticed this and scrapped the bounty, the breeders released their now worthless animals. In the end, there were more snakes than before. This is exactly the pattern the term refers to: a rule rewards a measurable quantity, and people satisfy the measurement instead of solving the actual problem.

When the metric crowds out the goal

Whether the cobra story is historically accurate is disputed. The pattern behind it is not. Every organization has to measure success somehow, and every measurement is a simplification. As soon as money, promotions, or ratings are tied to that number, everyone aligns themselves with the number. The number rises, the actual intended goal falls behind.

Economists know this as Goodhart’s Law: a metric stops being a good measure the moment it becomes the target. An example from everyday school life: if a school is evaluated solely on average grades, it can sort out weak students instead of teaching better. The metric improves, the quality of education does not. The cobra effect is the especially drastic variant of this, in which the situation actually gets worse.

For technology companies, this is not a marginal issue. Anyone who builds a system used by millions of people amplifies every design flaw a millionfold. A poorly chosen target metric then doesn’t act like a single mistake, but like a permanently miscalibrated machine.

Why software is especially prone to this

A computer program has no sense of the meaning behind an instruction. It optimizes exactly the quantity it has been given, and nothing else. In learning systems, this specification is called the objective function: a calculation rule that defines what counts as success. The system then searches systematically for the path that increases this number the most — including a path nobody had thought of.

Experts call this reward hacking, meaning the gaming of the reward. In a well-known experiment, a piece of software was supposed to win a boat race. Points collected along the way were rewarded. The program discovered that it could drive in circles and collect the same points over and over again. It achieved high scores and never finished the race. The specification was fulfilled, the goal was missed.

A common mistake is to consider this a programming error. It is not. The system works exactly as described. What was wrong was the description. That is why, in AI safety research, the rule holds that target specifications should never be taken at face value without checking how they could be exploited.

Recommendation algorithms, chatbots, and bonus programs

The effect is most visible in recommendation systems, i.e., the software that decides which video or post is shown to you next. For a long time, the central target metric was watch time. As a result, the system learned which content kept people engaged the longest. Often this is outrageous or extreme content. The target metric rose, user satisfaction fell. Several platforms have since changed their target metrics.

The pattern also appears in language models. Among other things, these systems are trained to give answers that people rate highly. One side effect: the model learns to sound agreeable and confident, because that goes over well. Whether the answer is actually correct is not directly checked by the reward. This gives rise to polite, convincingly phrased false information.

In business news, you mostly encounter the term in the context of incentive systems. Bonuses for closed contracts lead to contracts nobody needs. Bonuses for quickly closed support tickets lead to tickets being closed without a resolution. The usual countermeasure is to look at several metrics at once and have samples reviewed by humans. A single number can almost always be gamed; a bundle of numbers is significantly harder to game.