Token Economics

Programs like ChatGPT don’t read text as letters, nor as whole words. They break it down into small building blocks called tokens. A token is usually a syllable, a short word, or part of a word. As a rough rule of thumb: 100 tokens correspond to about 75 English words, and somewhat fewer in German. These building blocks are exactly the computational unit that providers work and bill with. Token economics is the examination of what such a building block costs, how many of them accumulate, and who ultimately pays for it.

Why every answer has a price

For users, an AI answer feels free. For the provider, it never is. Every building block has to run through specialized computing chips, which consume electricity and processing time. With millions of users, tiny amounts add up to considerable sums.

That’s why token economics is one of the most important figures in the AI business. Providers publish prices per million tokens, often split between incoming and outgoing text. Outgoing text is typically more expensive because the model has to generate it itself, building block by building block. Incoming text, on the other hand, is only read and processed once.

These prices have fallen dramatically within a few years, in some models by more than 90 percent. At the same time, requests have gotten longer, because users copy entire documents into them. A company’s bill can therefore rise even though the price per building block falls. A common misconception is that cheaper models automatically mean cheaper operation.

How the building blocks add up

In a conversation with a chat program, it’s not just your latest question that gets billed. The model has no memory in the human sense. So that it knows the conversation history, the entire previous history is sent along again with every new question. A long conversation therefore becomes more expensive with every round than the previous one.

On top of that come instructions the user never sees. Companies often prepend several pages of rules to their models, the so-called system prompt. Search results or excerpts from company-internal documents are also sent along. In some applications, these hidden building blocks make up the largest part of the bill.

There are several levers against this cost spiral. A cache ensures that identical text beginnings are only computed once and are cheaper afterward. In addition, many companies send simple requests to a small model and only difficult ones to a large one. This is called routing, and it often lowers costs significantly without users noticing any difference.

Token prices in the news and in products

In the quarterly figures of technology corporations, tokens now appear as their own metric. Companies state, for example, how many trillions of tokens they process per month. Analysts read from this how strongly demand for AI is really growing. And investors ask whether revenue per building block covers the electricity bill.

Anyone building software themselves encounters token economics directly. Access to a model runs through a programming interface, and billing there is done by building blocks. Paid subscriptions for end customers are also derived from this: the monthly price is a bet that the average user consumes fewer building blocks than their subscription fee covers. Heavy users therefore eventually hit upper limits.

This term should not be confused with tokens from the crypto world. There, a token denotes a digital unit of value on a blockchain — something completely different. In AI, a token is just a piece of text. When an article about computing costs or model pricing talks about tokens, it is almost always about the meaning described here.