State

State simply means “Zustand” in German. What’s meant is everything a running program remembers about what has already happened. A shopping cart in an online store is a good example: it knows which items you clicked on earlier. Without this remembered state, every click would be a fresh start. Programs that remember something are called stateful. Programs without a memory are called stateless.

Why a chatbot without state would just be a calculator

The difference between stateful and stateless determines what a system can actually accomplish. A stateless system treats every request in isolation. It’s like a calculator: you enter something, get a result, and afterwards everything is forgotten. For many tasks, that’s completely sufficient.

But as soon as several steps belong together, state is needed. A chat only works if the system knows what you two were talking about two sentences ago. The question “And how old is he?” makes no sense on its own. Only the stored history turns it into a meaningful question. This is exactly why state is a central topic for AI assistants.

But state also has a price. Whoever remembers something has to store and protect it somewhere. Stateless systems, on the other hand, are easy to multiply: you simply start ten identical copies, and it doesn’t matter which one handles your request. With stateful systems, this isn’t so simple, because each copy knows different things.

Where the memory actually lives

Technically, state can live in very different places. In the program’s working memory, it’s available the fastest, but disappears on restart. In a database, it survives even a crash, but is slower to retrieve. Many systems combine both.

With language models, the situation is particularly interesting. The model itself is, at its core, stateless. It receives a text and predicts the next word, nothing more. The memory arises around it: the software sends the conversation history so far along with every request. This text that gets sent along is called the context window.

A common misconception is therefore that the model remembers you. In fact, it reads the transcript from scratch every single time. If the context window is full, the beginning of the conversation drops off. This is exactly what you experience when a chatbot suddenly forgets earlier details after a long conversation.

From save files to AI agents

You encounter state constantly in everyday life, usually without the name. The save file in a video game is a state. The cookie that keeps you logged into a website is too. The history in a music app also belongs here. It’s always the same question: what does the system still know from before?

In tech news, the term comes up especially with so-called AI agents. These are programs that independently work through a task over many steps. Such an agent has to keep track of which steps have already been completed and what came out of them. This working state is often the trickiest part of the whole system.

State should be distinguished from the model weights, i.e. the numbers learned during training. The weights are long-term knowledge and don’t change during operation. State, by contrast, is short-term and belongs to exactly one running session. Once you end the chat, it’s generally gone, while the model remains unchanged.