Output token
An output token is a small piece of text that an AI language program generates as part of its response – usually a word, part of a word, or a punctuation mark. Because such programs build their answer piece by piece, output tokens become the most important unit of measurement for speed and cost.
Programs like ChatGPT don’t write their answer all at once. They assemble it from many small pieces of text, one after another. Such pieces are called tokens. A token is often a whole short word, sometimes just a syllable, sometimes just a comma. All the pieces that the program itself generates and outputs are called output tokens. The pieces that go in beforehand – that is, your question – are called input tokens instead.
The currency in which AI providers bill
Anyone using an AI model via a programming interface, i.e. automatically from their own software, pays per token. Providers typically state their prices per one million tokens. In doing so, output tokens almost always cost more than input tokens, often three to five times as much. The reason is the computational effort: text that is read in can be processed in large blocks in parallel, whereas each output token has to be computed individually.
For companies, this quickly becomes a central cost issue. A customer service bot handling a hundred thousand conversations a day generates millions of output tokens. That’s why AI systems are often instructed to keep it short. A concise answer is not only more pleasant to read, it’s also cheaper.
Output tokens are also the yardstick for speed. A model’s speed is measured in tokens per second. About 20 tokens per second feels fluent to a human, because that’s roughly how fast people read. When a chatbot appears word by word in the browser, you are literally watching the output tokens come into being.
How one token after another comes into being
At its core, the model always does the same thing: it looks at the text so far and estimates which token fits best next. To do this, it calculates a probability for thousands of possible continuations. Then it picks one, appends it to the text, and starts over. This process repeats until the answer is finished or a set upper limit is reached.
What matters here is that every newly generated token becomes part of the input for the next step. So the model is constantly reading its own half-finished answer along with everything else. That’s precisely why this phase can hardly be parallelized. The tenth sentence can only be computed once the ninth one is in place.
A common misconception is that a token is the same as a word. In German, that’s rarely true. Long compound words like “Krankenversicherung” are split into several pieces. As a rule of thumb: a hundred German words correspond to roughly 130 to 170 tokens. English texts need fewer tokens for the same content, which is why English is cheaper with some providers.
Where you’ll encounter token figures
In price lists from providers such as OpenAI, Google, or Anthropic, input and output tokens are always listed separately. In technical descriptions you’ll also often find a maximum output length, say 8,000 or 64,000 tokens. It determines how long a single answer can be at most. If an AI cuts off mid-sentence, this limit has usually been reached.
The term also crops up in stock market and business news. Companies cite the number of tokens processed per month as a growth metric, similar to how other companies cite user numbers. Analysts factor in token prices when estimating whether an AI business model can be profitable.
Output tokens are especially relevant for so-called reasoning models. These models generate a long internal chain of thought before the actual answer. This chain of thought also consists of output tokens and is billed as well, even if you never get to see it. A seemingly short answer may thus have cost thousands of tokens.
