Connectionism

Connectionism is a school of thought about how thinking comes about at all. Its basic assumption: intelligence does not arise from a collection of rules, but from the interplay of very many tiny computing elements. Each individual element can do almost nothing. It takes in numbers, processes them, and passes on a new number. Only the dense network of millions of such connections produces achievements that we call knowledge or ability. The knowledge is not stored at any particular location, but in the strength of the connections themselves. The name comes from exactly this idea: connection means Verbindung in German.

The break with rule-based AI

Until the 1980s, a completely different notion dominated. The aim was to build intelligence out of explicitly formulated rules: if this case occurs, then do that. Such systems are called symbolic AI. They worked well for chess or tax calculations, that is, wherever everything could be written down cleanly.

For everyday tasks, however, they failed. Nobody can put into rules how one recognizes a face or why a sentence sounds odd. Connectionism turns the matter around. Instead of specifying rules, the system is shown examples. It adjusts its connections until it handles the examples correctly. This is called training.

It is precisely this approach that has prevailed. Image search, language translation, and chatbots are all based on it. The price for this, however, is high: with a connectionist system one can hardly look up why it gave a particular answer. With a rule-based system, one could simply look up the responsible rule.

Connections, weights, and learning

The computing units in such a network are called artificial neurons because they roughly resemble nerve cells. Each connection between two neurons carries a number, the so-called weight. A high weight means: the signal from before counts a lot. A weight close to zero means the connection plays practically no role. The totality of all weights is what the network has learned.

At the start, these weights are set randomly, so the network’s answers are nonsense. During training, the output is compared with the correct solution and the error is measured. Afterwards, all weights are shifted a small step in the direction that reduces the error. This process is repeated millions of times. After enough iterations, the network delivers usable results.

A common misconception is that such a network works like a real brain. The similarity is only a rough borrowing. Biological nerve cells are considerably more complicated, and the brain does not learn by adjusting itself millions of times against training examples. Connectionism is inspired by biology, but it does not copy it.

What of this is found in today’s products

Practically every AI product mentioned in the news is built in a connectionist way. The language models behind chatbots are huge networks with many billions of weights. When a report speaks of a model with 70 billion parameters, it is precisely these connection strengths that are meant. Facial recognition on smartphones and recommendations from streaming services are also based on this.

The term itself is mostly found in texts about the history or foundations of AI. It barely appears in product marketing, where it is simply called neural network or deep learning. However, anyone who knows about connectionism understands why many AI systems behave the way they do: they recognize patterns in data rather than follow rules.

This also gives rise to the typical weaknesses that are reported on. Such a system can confidently give wrong answers because it does not check against a set of rules, but continues a plausible pattern. And it absorbs biases from its training data, because that is exactly what its entire knowledge consists of.