Bitter Lesson

The Bitter Lesson is a thesis about the history of artificial intelligence. The Canadian researcher Richard Sutton formulated it in 2019 in a short text. His observation: for decades, researchers tried to program their own knowledge about a problem directly into computers. They wrote down, for example, which chess positions are good or how German grammar works. In the end, however, the programs that instead learned on their own from huge amounts of examples, while consuming enormous amounts of computing time, always won. Sutton calls this lesson bitter because it is humbling for researchers: their carefully built expert knowledge was ultimately overtaken by sheer computing power.

Why the lesson hurts so much

Sutton cites several examples. In chess, many researchers relied on teaching the computer human patterns of thought. But it was Deep Blue that won in 1997, a program that mainly relied on brute-force calculation of enormous numbers of moves. Something similar happened in Go and in speech recognition. In both fields, hand-built, knowledge-based systems lost out to methods that learned from data.

According to Sutton, the reason lies in costs. Computing power becomes ever cheaper over the decades, and dramatically so. Human intellectual labor does not become cheaper. A method that automatically benefits from faster chips thus grows over time. A method that relies on built-in human knowledge stalls as soon as that knowledge has been written down.

This is why the Bitter Lesson is more than a historical anecdote. For many companies it serves as a guideline for action. When choosing between two approaches, one should pick the one that becomes ten times better with ten times more computing power. This explains part of the billions of dollars being invested in data centers.

Searching and learning instead of writing rules

Sutton names two methods that can be fed with arbitrary amounts of computing power. The first is search: the program tries out a great many possibilities and evaluates them. The second is learning: the program adjusts its internal numerical values against millions of examples until its predictions are correct. Both methods simply get better with more chips and more time, without anyone having to formulate new rules.

A comparison makes this clear. Imagine two cooks. One learns a hundred fixed recipes from a master and cooks them perfectly. The other tries out a thousand variations every day and keeps whatever tastes good. In the short term, the first is better. As soon as the second gets a kitchen with a thousand stovetops, he pulls ahead.

An important distinction should be made. The Bitter Lesson does not claim that thinking is pointless. It distinguishes between two kinds of research work. Building in finished knowledge about the world brings little benefit in the long run. Developing better learning methods and better search methods brings a great deal, because every additional hour of computing benefits from it.

The thesis in current AI debates

In news about AI, the Bitter Lesson frequently comes up as an argument. When a language model like GPT gets better mainly by being made larger and trained on more text, observers point to Sutton. AlphaGo and its successor AlphaZero are also considered prime examples. AlphaZero learned Go purely through games against itself, without any human games, and became stronger than the version with human prior knowledge.

Conversely, the thesis is also cited as a warning. Anyone who builds an AI product on hand-written rules risks that a larger model will soon solve the same task on its own. Such companies are considered vulnerable in the industry, because the next leap in models can wipe out their distinguishing feature.

However, the Bitter Lesson is not without controversy. Critics point out that computing power, electricity, and training data are finite and cannot keep growing this fast forever. Moreover, many human decisions are still embedded in today’s models too, for instance in the selection of training data or in the fine-tuning of responses. The thesis is thus more of a rule of thumb for long-term trends than a law of nature.