Token-based costs

When a company integrates an AI text service like ChatGPT into its own software, it doesn’t pay per month and not per user. It pays according to the amount of text. To do this, every text is broken down into small building blocks, so-called tokens. A token is usually a short word or part of a word, in English on average about four characters. The provider counts how many such building blocks go in and how many come back out, and bills for both. This is exactly what the term token-based costs refers to.

Why a bill is rarely predictable

With classic software, you pay a fixed amount per month. The costs are fixed in advance, no matter how intensively you use the program. With AI services, it’s different: every single request costs money, and a different amount each time. A short question costs fractions of a cent, analyzing an 80-page contract costs significantly more. For companies, this is an unfamiliar metric to calculate with.

The problem intensifies when many people use the service. A chatbot on a website with a million visitors generates millions of requests. Small amounts then add up to six-figure bills. That’s why companies carefully examine whether an AI feature is even worthwhile, and continuously monitor consumption.

The metric is also important for investors and journalists. If a provider lowers the price per million tokens, this changes the economics of entire business models. Such price cuts have regularly been a topic in tech news in recent years, because they make applications possible that were previously too expensive.

How the price of a request is composed

The billed amount has two parts. The input encompasses everything the model gets to read: the user’s question, attached documents, and hidden instructions from the operator. The output is the text the model generates. Output tokens are usually two to five times more expensive than input tokens, because the model has to compute them one after another individually.

Prices are usually quoted per million tokens, for example in the form “$0.50 per million input tokens.” As a rough guide: one A4 page of text corresponds to roughly 700 tokens. A thick book comes to several hundred thousand tokens. Anyone who has such a book summarized by an expensive model quickly pays several euros for it.

A common misconception is that only the current question counts. In an ongoing conversation, the software usually resends the entire previous history as well, because the model itself has no memory. A long conversation therefore becomes more expensive with every message. Countermeasures include caches for recurring text segments, shorter instructions, and choosing a smaller model for simple tasks.

Where the billing becomes visible

You encounter it most directly in providers' price lists. OpenAI, Anthropic, Google, and Mistral all publish tables with amounts per million tokens, broken down by model as well as by input and output. Anyone with an account there sees their own consumption in an overview – similar to an electricity bill.

You also encounter the model indirectly as a regular user. Free chat offerings work with shorter responses or weaker models, because every request costs the provider real money. Limits such as “30 messages per three hours” also stem from this background.

In business news, the term comes up when discussing the margins of AI companies. Token-based costs must be distinguished from training costs: training a model is a one-time, very large investment. Token costs, on the other hand, recur with every use afterward and determine in the long run whether an AI product turns a profit.