Neural Network

A neural network is a computational method that learns from examples. Classical software follows rules that humans have written down in advance. A neural network, by contrast, is presented with a very large number of examples: for instance, millions of photos, each labeled with what is shown in it. From these examples, it adjusts its own internal numerical values so that it gets the right answers as often as possible. The name comes from a loose resemblance to the human brain, in which nerve cells pass signals to other nerve cells. This resemblance is very rough, though: internally, nothing happens except multiplying and adding, just billions of times over.

Why learning systems have outpaced rules

Many tasks cannot be written down as a set of rules. Try to explain in words how you recognize a cat in a photo. Every rule you come up with has counterexamples: cats seen from behind, in the dark, or half hidden. This is exactly where classical programs fail, and exactly where neural networks succeed.

Until around 2012, neural networks were considered an interesting but impractical idea. Then three things came together: vast amounts of digital data, significantly faster graphics chips, and better training methods. Since then, neural networks have become the best known method in almost every discipline. Speech recognition, translation, medical image analysis, and chatbots are all based on them.

Economically, this is why chipmakers like Nvidia have become so important. Neural networks require enormous amounts of computing power, both during training and later during operation. Whoever sells or rents out this computing power profits from the entire AI boom.

Layers, weights, and training

A neural network consists of layers of artificial neurons. A neuron is just a small computational cell: it takes in numbers, weights each of them, and passes on a new number. The input is first converted into numbers, for example an image into the brightness values of its pixels. These numbers travel layer by layer through the network until a result emerges at the end.

What matters are the weights, i.e. the factors by which each neuron multiplies its inputs. At the start they are random, so the network is essentially guessing. During training, each answer is compared with the correct solution and the error is calculated. Afterward, all weights are shifted a tiny bit in the direction that reduces the error. This process is called backpropagation and is repeated billions of times.

A common misconception: the network does not store examples. After training, only the weights remain, often billions of numbers. These are called parameters, and their number appears in model names like “7B” for seven billion. Because no one can tell what a single number means just by looking at it, neural networks are considered hard to interpret. Experts speak of a black box.

From phone photos to stock market news

You use neural networks several times a day, usually without noticing. Facial recognition for unlocking your phone is one example. So are speech recognition when dictating, autocorrect, music recommendations, and the spam filter in your inbox. Translation services and traffic forecasts in map apps work this way too.

In the news, you often encounter the term in disguised form. When deep learning is mentioned, it refers to especially deep neural networks with many layers. A transformer is a particular architecture that is especially well suited to language. And a large language model like the one behind ChatGPT is nothing other than a very large neural network trained on text.

For investors, the term therefore serves as a signal for which part of the value chain a company belongs to. Some supply the chips for training, others the data centers, and still others the finished models or applications built on top of them. Anyone reading a report about AI should check which of these roles is being referred to. The difference determines who ultimately makes money.