Stateless

Stateless means “without state.” It refers to a computer program that has no memory between two requests. It processes each request on its own and then forgets everything again afterward. Anyone who wants something from such a program therefore has to supply all the details every single time. The opposite is called stateful, meaning it holds state: there, the program remembers who you are and what happened before. Picture a government office where every clerk treats you like a first-time visitor. You have to present your ID and all your documents every single time. Inconvenient for you, but every clerk can serve you without needing to know your file.

Why stateless services can be scaled up endlessly

Large internet services don’t run on a single computer but on thousands of them. A load balancer sends each incoming request to a free machine. If no machine needs to remember anything, it doesn’t matter at all which one gets the request. That’s exactly what makes growth easy: if you need more capacity, you simply add more identical machines.

With stateful services, this is more cumbersome. There, your conversation history sits on a specific machine. If that machine fails, the history is gone or has to be migrated first. All your requests also have to be routed to the right machine. Such rules take effort and tend to go wrong more often.

The price of statelessness is repetition. If every request carries along the complete context, the amount of data grows. For AI services this can get expensive, since billing is often based on the amount of text. Still, the approach is usually the better choice, because it makes outages and errors less frequent.

Where the context gets moved to

Stateless doesn’t mean that information disappears. It’s simply kept somewhere else, namely not within the program itself. Two locations are common: with the user, or in a shared database that all machines can access. The server itself remains a pure computing workhorse without memory.

A well-known example is the token. This is a small, tamper-proof encrypted file that your browser stores and sends along with every request. It states who you are and what you’re allowed to do. The server only checks its authenticity and doesn’t need a login list in its own memory. After the response, it knows nothing about you again.

AI chats work in a similar way. The language model itself doesn’t remember your last conversation. The chat app collects the history and sends it along in full with every new question. That’s why a chatbot seems to have a memory, even though the model itself operates statelessly. And that’s why the thread breaks off when the history gets too long and has to be trimmed.

Stateless in cloud bills and product descriptions

The term shows up everywhere software is offered over the internet. Programming interfaces, through which programs talk to one another, are almost always built stateless. In technical descriptions you’ll then read phrases like “stateless API.” That’s a quality promise: the service can handle many users at the same time.

In business news, statelessness is a cost topic. For stateless services, cloud providers often only bill actual compute time, since nothing has to run continuously. Anyone who wants to store state, on the other hand, pays extra for databases and storage. For AI providers, this principle also explains why long conversations drive up the bill.

A common misconception: stateless does not mean “doesn’t store any data about you.” A stateless service can very well log everything and store it in databases. It’s purely a technical question of where the connection between two requests resides. Data privacy is a different matter and is not automatically improved by statelessness.