Emergence

Water consists of individual molecules. No single molecule is wet. The thing only becomes wet once a great many of them come together. Such properties, which arise from the interplay of many parts and cannot be attributed to any single part, are called emergent. In AI research, emergence means exactly that: a system can suddenly do something that cannot be read off from its individual components. A language program, for example, is only trained to guess the next word in a text. Yet a large model can then suddenly solve arithmetic problems or translate from German into Japanese. No one specifically showed it how to do that.

Abilities nobody ordered

Emergence is one of the reasons why the development of AI is hard to plan. With ordinary software, what it should be able to do is fixed in advance. A calculator computes because someone programmed the rules of arithmetic into it. A large language model, by contrast, is not given a rulebook but huge amounts of text. What it ends up being capable of only becomes apparent after training.

For companies, this is both an economic risk and an opportunity at the same time. A training run can cost many millions of euros, and the outcome cannot be predicted exactly beforehand. Sometimes an ability emerges that makes an entirely new product possible. That is exactly what happened several times in the leap from older to newer model generations.

The flip side concerns safety. If no one can predict what a system will be capable of, then unwanted abilities cannot be reliably ruled out either. That is why major providers test their models for months before release. In doing so, they specifically look for behavior they did not expect.

The leap at the size threshold

Emergent abilities are strongly tied to the size of a model. Size here is measured in parameters. These are the adjustable numerical values inside the system, comparable to millions of tiny dials. A small model has a few billion of them, a large one several hundred billion. Added to this are the amount of training text and the computing time expended.

The typical pattern is interesting. On a task like three-digit mental arithmetic, small models fail completely. Somewhat larger ones still hover near zero percent correct answers. Beyond a certain size, the success rate then jumps sharply upward. So the curve does not rise gently but bends sharply.

Whether this leap really happens inside the model is disputed. Researchers at Stanford University showed in 2023 that the effect is often due to the measurement method. If you only score an arithmetic problem as right or wrong, you see a jump. If you count how many digits are correct, you see a gradual increase. Emergence is therefore also a matter of perspective.

Emergence in headlines and products

In the news, the term usually comes up when new models are introduced. It is then said that a system shows surprising or unexpected abilities. This almost always refers to emergence. It often concerns programming, logical reasoning, or understanding multi-step instructions.

In everyday life, you encounter the result without anyone using the word. A chatbot that writes a poem in the style of a particular author was not specifically trained to do that. The same applies to models that can explain a bug in program code. These abilities are byproducts of general training.

A common misconception is that emergence is a sign of consciousness or genuine understanding. It is not. The term merely describes the fact that an ability appears unexpectedly. It says nothing about what is actually going on inside the system. Even an anthill appears to act according to a plan, even though no single ant knows a blueprint.