Hidden Layer

Many AI systems consist of what is called a neural network. This is a computational model that sends numbers through many successive stages. Right at the front sits the input, for example the pixels of a photo. Right at the back sits the result, such as the answer “cat”. All the stages in between are called hidden layers. They are not visible from the outside, and it is within them that the actual work takes place.

Why deep networks can do more than shallow ones

A network with no hidden layer at all can only represent very simple relationships. It might, for instance, decide whether an image is overall bright or dark. That is not enough to answer whether a face can be seen in it. Only several stages in succession make it possible to combine simple observations into more complicated concepts.

This is precisely where the term “deep learning” comes from. Here, deep simply means: many hidden layers. Early networks from the 1990s often had only one or two. Today’s language models reach several dozen up to over a hundred. This depth is one of the main reasons AI systems have become so much better since around 2012.

However, more layers are not automatically better. Very deep networks are harder to train, because error information weakens on its long path forward. Researchers have developed tricks for this, such as shortcuts that route signals past several layers. Without such aids, depth tends to be more of a problem than an advantage.

From pixel to meaning: what emerges within the stages

Each hidden layer consists of many small computational units, often called neurons. Each neuron receives the numbers from the previous layer, weights them, and adds them up. These weights are the model’s actual adjustable parameters; they are adjusted during training. After that, the result still passes through a simple transformation that prevents the entire network from becoming just one big calculation.

In image recognition, one can observe quite clearly what results from this. The first hidden layers respond to edges and color transitions. Middle layers assemble these into shapes, such as circles or striped patterns. Later layers respond to whole object parts like eyes or wheels. It is like reading: first letters, then words, then the meaning of the sentence.

An important, common misunderstanding: no one specifies which layer takes on which task. This division of labor arises on its own during training. That is why the values in a hidden layer are, at first, for humans just a long string of numbers with no recognizable meaning. A dedicated field of research, interpretability, attempts to decipher these numbers after the fact.

Hidden layers in products and headlines

In the technical descriptions of AI models, the number of layers almost always comes up. One provider, for example, states that its model has 80 layers and 70 billion parameters. The parameters here are precisely those weights contained within the hidden layers. Such figures give a rough sense of how large and how costly a model is to run.

One also encounters the term more often in practice than one might think. When a company adapts a finished model to its own task, it often freezes the earlier hidden layers and retrains only the later ones. This saves an enormous amount of computing time, because the general basic patterns have already been learned. Search functions work similarly, reusing the numerical state of a middle layer as a compact description of a text or image.

A distinction helps when reading news. Not every layer in a software diagram is a hidden layer in the sense meant here. In IT, one also speaks of layers when programs build on top of one another, such as database, logic, and interface. In AI texts, however, what is almost always meant is the computational stage inside a neural network.