Token Prices

When a computer program processes text, it first breaks it down into small chunks. These chunks are called tokens. A token is often a short word, sometimes just a syllable or a punctuation mark. As a rule of thumb: 1000 tokens correspond to roughly 700 words in English. AI service providers bill their usage precisely in this unit. The token price therefore tells you what a thousand or a million of these chunks cost.

Why cent amounts turn into real bills

Looked at individually, the amounts seem laughably small. A provider might charge 3 dollars for a million input tokens. A normal chat question thus costs a fraction of a cent. But this very smallness is deceptive once many requests add up.

A company that automatically answers a hundred thousand customer inquiries every day quickly ends up paying five-figure sums by the end of the month. That’s why token prices are not a side issue for companies, but a central item in the calculation. Whether an AI product pays off at all often hinges on exactly these numbers. Anyone earning two euros per customer but spending three euros on tokens loses more money with every new user.

On top of that comes the price collapse. Costs per token have fallen dramatically in recent years, in some cases by more than a factor of ten per year. Applications that were unaffordable in 2023 are commonplace today. When business news talks about a price war between AI providers, it is almost always about token prices.

How a bill is put together

Almost all providers distinguish two directions. The text you send in is called input. The text the model generates is called output. Output is significantly more expensive, often three to five times the price of input. The reason lies in the computational work: the model generates each output token one after another, individually, while it can read in the input all at once.

An example: you send in a ten-page contract, around 8000 tokens, and get back a summary of 500 tokens. At 3 dollars per million input tokens and 15 dollars per million output tokens, that costs about 3.2 cents. It gets interesting when the same request runs a thousand times a day.

A common misconception: you don’t just pay for your own question. In a longer conversation, the program usually resends the entire previous history so the model knows the context. That’s because models have no memory between two requests. Long conversations thus become more expensive with every round. Some providers cushion this with discounts when the same starting text occurs multiple times.

Token prices in products and headlines

As a private user, you rarely see token prices directly. A chatbot subscription for 20 euros a month hides the billing behind a flat fee. The prices only become visible in providers' price lists for developers. There, for each model, it states what a million input and output tokens cost.

Token prices regularly come up in the news whenever a provider introduces a new model. The message is usually: same quality, half the price. The debate around Chinese providers also revolved heavily around this point, because their models were offered at a fraction of the usual prices.

It’s important to distinguish this from training costs. Training a model might cost hundreds of millions of dollars as a one-time expense. Token prices, by contrast, concern ongoing operation, i.e. the use of the finished model. Both together determine whether an AI provider makes money in the long run.