Cache Discount
A cache discount is a price reduction that AI service providers grant when part of the input has already been sent before and is stored in a cache. Because the computer doesn't need to reprocess this part, it often costs only a tenth of the normal price.
Anyone using an AI like ChatGPT not through the website but directly from their own program pays per request. Billing is based on the amount of text: everything you send, and everything that comes back. Now, programs often send the same block of text along again and again, for example a long set of instructions on how the program should behave. Providers therefore cache this recurring beginning instead of recalculating it completely each time. The cache discount is the price reduction you get for this. The term cache here simply means a temporary store: a place where something is kept ready that you’ll need again shortly.
What the discount changes on the bill
The savings are not pocket change. Depending on the provider, cached input text costs only ten to fifty percent of the normal price. With some services it’s even just five percent. Anyone making millions of requests daily saves very large amounts this way.
The effect is especially strong for applications with a long, fixed preamble. A customer service bot might receive 8,000 words of company rules and product information along with just a single short customer question. The fixed part then makes up over 95 percent of the input. It is precisely this part that becomes cheaper.
Besides money, the cache also saves time. The first response arrives noticeably faster, because the computer doesn’t have to analyze the known part again. With very long preambles, the wait time can be cut in half. For developers, the cache discount is therefore both a pricing and a speed question.
Why only the beginning counts
A language model doesn’t process text as a whole, but piece by piece from front to back. In doing so, intermediate results emerge, a kind of notes on each piece of text processed so far. These notes always depend only on the text before them, never on the text after. This is exactly why they can be reused: if two requests begin identically, the notes for this beginning are identical too.
The cache thus stores the intermediate results for a prefix, meaning the shared beginning of several requests. For the next request, the provider checks how far the match extends. Up to that point, it loads the stored notes. Only from the first differing character onward is anything actually computed.
This leads to an important rule of thumb for practice: fixed content belongs at the front, variable content at the back. If the current date sits right at the beginning, the text changes every day and the cache never kicks in. Another common misconception is that the cache lasts forever. In fact, it is usually deleted after a few minutes if nobody uses it.
Where cache discounts show up in price lists
In the pricing tables of OpenAI, Anthropic, or Google, you now find three numbers instead of two. Alongside the price for input and output, there’s a separate line for cached input. Sometimes it’s called cached input, sometimes cache read. Anyone comparing offers should read this line too, or they’ll end up calculating too high a price.
With some providers, the caching itself costs money. The first write into the cache then costs somewhat more than a normal request, while every reuse costs significantly less. This only pays off from a certain number of repetitions onward. Other providers activate caching automatically and charge nothing for writing.
In the news, the term usually appears in connection with falling AI costs. When companies report that running their assistants has become 80 percent cheaper, a cache discount is often behind it. It should not be confused with the batch discount: that applies to requests whose response you pick up hours later. Both lower the price, but through completely different mechanisms.
