Multilayer Perceptron

A Multilayer Perceptron is a computational structure made of layers. Each layer consists of individual units called neurons — named by analogy with nerve cells in the brain. Each of these units takes in numbers, performs some computation on them, and passes on a new number. The special feature: the units of one layer are connected to all units of the next layer, which is why it is also called a “fully connected network.” Input data comes in on the far left, output comes out on the far right — in between lie one or more so-called hidden layers, which carry out the actual processing. The direction is always the same: from left to right, without loops or feedback paths.

The role of the MLP in modern AI systems

The MLP is not a historical relic — it is embedded as a core building block in almost every modern AI model. Language models such as GPT or Gemini consist largely of MLP blocks arranged alternately with other components. Without the MLP, these systems would not exist.

The reason lies in what an MLP is capable of: it can recognize arbitrarily complicated patterns in data — provided it has enough layers and enough neurons. Expressed mathematically, a sufficiently large MLP can approximate any function. That sounds abstract, but in practice it means: whether speech recognition, image classification, or text translation — an MLP can, in principle, learn the necessary relationships.

However, the MLP alone is not the most efficient solution for many tasks. For images, specialized networks that exploit spatial structures work better. For language, additional mechanisms are needed that take sequence order into account. The MLP provides the foundation, but real systems build on top of it.

Weights, activation, and training

The connections between neurons each have a strength — a so-called weight value. This value determines how strongly a signal is passed from one unit to the next. During training, it is precisely these weights that are adjusted step by step: the network is given an example, makes a prediction, compares it to the correct answer, and then adjusts all weights by a small amount. This process repeats millions of times.

So that the network does not learn only simple linear relationships — that is, not just lines and planes — a so-called activation function is applied after each layer. It ensures that the model can also represent nonlinear, meaning curved or more complex, patterns. Without this step, a network with a hundred layers would be just as limited as one with only a single layer. The activation function most commonly used today is called ReLU: it passes positive numerical values through unchanged and sets negative ones to zero — simple, but effective.

A typical small MLP for an image task might have three to five layers with a few hundred neurons each. Large language models, on the other hand, contain MLP blocks with billions of weights. The difference lies not in the principle, but only in the scale.

MLP in products and headlines

When tech media write about “neural networks,” they almost always mean systems built on the MLP principle. ChatGPT, Google's Gemini, Meta's Llama — all of them use MLP layers as one of their central building blocks. So anyone reading reports about language models, image generators, or AI chips indirectly encounters the MLP in every article.

In the chip industry, too, the MLP plays a direct role. Nvidia's graphics processors were originally developed for gaming, but turn out to be excellently suited for carrying out the many parallel computations an MLP requires. Nvidia’s enormous success in recent years is directly tied to the fact that MLP computations run especially fast on their hardware.

The MLP also shows up in embedded systems: voice assistants on smartphones, fraud detection at banks, or automated quality control in factories often rely on lean MLP variants that get by with little computational overhead. The underlying principle is always the same — only the scale and the place of application change.