Sequential Model

A Sequential Model is a specific architectural form of programs that learn from examples. Such learning programs consist of several computational stages called layers. Each layer takes in numbers, processes them, and passes new numbers along. In a Sequential Model, these layers are lined up like beads on a string: an input, then layer after layer, and finally a result. There are no branches and no paths back. The English name means exactly that: sequential, i.e., strictly in order.

The simplest architecture as a starting point

Anyone learning about neural networks almost always starts with this architecture. The reason is practical: you only need to specify a list of layers, nothing more. The order in the list is also the order of computation. This means you can build a functioning model in just a few lines of code.

Despite its simplicity, a Sequential Model is no toy. For many standard tasks, it is entirely sufficient. A model that recognizes handwritten digits can be built this way. So can a prediction of whether a customer will cancel a contract — this structure suffices.

It is important to distinguish this from a related term. In natural language processing, people speak of sequence models when a model processes sequences of words. That means something different. A Sequential Model is about the structure of the network, not the type of data. This confusion is a common beginner’s mistake.

Layers on a string

You can picture the model like an assembly line. A raw material goes in, and each station processes it a bit further. At the end of the line stands the finished product. No station skips another, and none sends anything back.

Concretely, when programming, you specify a list. First comes a layer that takes in the input data, for instance the pixels of an image. Then follow several intermediate layers that work out patterns in the data. Finally comes an output layer with as many outputs as there are possible answers. For ten digits, that means ten outputs.

This clarity comes at a price. As soon as a model needs two different inputs at once, say an image and a text, this architecture fails. The same applies if a layer needs to pass its result to two places. For such cases, developers use what is called the Functional API. There, layers can be wired freely, including branches and merges.

From tutorial to production system

The term appears most often in programming libraries. Keras and Google's TensorFlow offer a class called Sequential, and PyTorch has a similar one named nn.Sequential. Anyone opening a neural network tutorial encounters it within the first few lines. It is also the standard choice in school projects and bachelor’s theses.

In the products of major providers, this architecture rarely stands alone. Modern language models like GPT or image models have branched structures with connections that skip individual layers. Such shortcuts are necessary for very deep networks so that training can succeed at all. There, a pure Sequential Model is too rigid.

Nevertheless, the architecture still appears there too, namely as a building block. Developers combine several layers into a Sequential block and then insert these blocks into a larger, branched architecture. This keeps the code organized. Anyone reading the term in a repository description should therefore not take it as a sign of a particularly simple system.