COGS (Cost of Goods Sold)
COGS stands for "Cost of Goods Sold" and refers to the costs that arise directly because a product is sold or a service is provided. At AI companies, these are mainly the compute costs incurred in data centers, which is why COGS has become the key metric for the question of whether AI businesses can be profitable.
When a company sells something, it directly incurs costs as a result. A baker needs flour, yeast, and electricity for the oven. These directly attributable costs are summarized under the English term “Cost of Goods Sold,” or COGS for short. In German this is roughly called Umsatzkosten or Wareneinsatz. Not included are expenses that arise independent of the quantity sold: office rent, advertising, engineering salaries. Subtracting COGS from revenue leaves gross profit – the money that must be used to pay for everything else.
What COGS reveals about the quality of a business model
The share of gross profit in revenue is called gross margin. It is one of the first figures investors look at. Classic software companies achieve 75 to 85 percent: an additional copy of a program costs virtually nothing. A car manufacturer tends to be at 15 to 25 percent, because every car consumes sheet metal, batteries, and manufacturing time.
High gross margins are so attractive because they scale. If revenue doubles, cost of goods sold barely doubles along with it. The additional dollar of revenue remains almost entirely intact and can fund research, sales, or profit. Companies with low gross margins, on the other hand, must grow enormously just to get anywhere near a profit.
This is exactly where COGS becomes interesting in AI. Every answer from a chatbot costs computing time on expensive specialized chips. These costs grow with every user request – so they behave more like flour for the baker than like a software copy. That’s why the gross margins of AI providers are often 40 to 60 percent instead of 80. Whether this will change is one of the open, billion-dollar questions facing the industry.
What flows into cost of goods sold at AI companies
The largest item is running the models in everyday operation, i.e., answering requests. Experts call this operating mode inference, as distinct from the one-time training. Added to this are the rental or depreciation of graphics chips, electricity, cooling, and fees to cloud providers. Costs for people who review content or provide support are also frequently booked here.
Exactly where the line is drawn is partly a matter of interpretation. Many companies do not count the costs of training a new model as COGS, but as research instead. Others spread them over several years. This makes gross margins poorly comparable between companies. Anyone reading quarterly figures should therefore check the fine print to see what was included.
Cost of goods sold is lowered using technical tricks. Smaller models for simple tasks, coarser storage of internal numbers, and caches for frequent requests save computing time. Proprietary chips instead of purchased ones also help. Every cent per request counts when billions of requests occur daily.
COGS in quarterly reports and price tags
In every income statement, COGS appears directly below revenue. At US companies, this line item is often found in quarterly reports as “Cost of revenue.” If gross margin falls compared to the previous year, analysts immediately ask for the reason on the earnings call. At tech companies, the answer since 2023 has often been: increased AI infrastructure costs.
Customers notice the effect too. When a provider lowers the price per million processed text tokens, it is usually passing on reduced internal compute costs. Conversely, high cost of goods sold explains why powerful models are only available without limits in the paid subscription. Free versions are pure expense for the provider, with no revenue.
A common misconception: COGS is not the same as profit or loss. A company can have a brilliant gross margin and still post deeply negative numbers because it spends a great deal on research and personnel. COGS only describes the first stage of the calculation – but it is the stage that shows whether a business is viable at all.