StackPerf
StackPerf refers to measuring the performance of an entire technology stack rather than individual components: it counts what ultimately reaches the user, not how fast a chip is on paper. Such holistic assessments today determine which AI systems are truly affordable and fast in actual operation.
When a computer program runs, many layers work together. At the bottom sits the hardware, the actual computing machine. Above that lie programs that distribute the computing tasks, and at the very top the application that the user sees. This stack of layers is called, in technical terms, a stack. StackPerf is the question of how well this entire stack works together — measured in time, cost, and energy per completed task. The term thus deliberately stands in contrast to figures that describe only a single layer.
Why a fast chip does not yet make a fast system
Manufacturers of computing chips advertise very large numbers. They state how many computing operations their chip can theoretically perform per second. These figures are honestly measured, but they only apply in the ideal case. In practice, an AI system often achieves only a fraction of that.
The reason lies in the layers above. The data must first be transported from memory to the chip. The software must sensibly distribute the work across many chips. If something goes wrong here, the expensive chip simply sits idle most of the time. It is precisely this waiting time that becomes visible when looking at the whole stack.
Economically, this is decisive. Anyone building a data center for AI spends billions on hardware. Whether this hardware is utilized at 20 or 60 percent changes the cost per response by a multiple. That is why investors and buyers today are more interested in overall figures than in chip data sheets.
What is actually counted in a stack measurement
Such a measurement always begins with a real task. One example: a chatbot is supposed to answer a thousand user questions. What is measured then is how long this takes, how much electricity is consumed in the process, and what the operation costs during that time. All layers are evaluated together, because the user also only experiences them together.
You can imagine it like a highway. Theoretical chip performance corresponds to the top speed of the cars. But what really matters is how many people arrive at their destination per hour. A traffic jam at the on-ramp makes fast engines worthless. A stack value measures the people arriving, not the engine power.
It is important to distinguish between two quantities. Latency describes how long a single request takes. Throughput describes how many requests are completed in total within an hour. Both can be traded off against each other: whoever batches many requests together increases throughput, but each individual response arrives later. A common mistake is to interpret a good number in one of these two quantities as good overall performance.
Where such overall figures appear in the news
They are most visible in comparative tests, so-called benchmarks. MLPerf, a series of standard tests for AI systems, is well known. There, complete setups of hardware and software compete against each other. When a manufacturer reports being twice as fast as the competition, the figure usually comes from such a test.
This also plays a role in business reports. Providers of cloud computing power, i.e. rented servers over the internet, present their customers with costs per million processed text tokens. These prices are essentially stack values, just expressed in euros. When they fall, it is often due to better software rather than new chips.
For you as a reader, the benefit is healthy skepticism. If a report only cites a single, very large number, the context is usually missing. Meaningful figures name the task, the complete setup, and the measurement conditions. If these details are missing, the number is more advertising than information.
