Scale-to-Zero
Scale-to-Zero means that a program on the internet is completely shut down as soon as nobody is using it, and only starts back up on the next request. This means no costs are incurred during idle periods, but the first request after a pause is noticeably slower.
Programs like websites or apps usually don’t run on your own computer, but on rented computers in large data centers. You normally pay for this rental per hour, regardless of whether anyone is visiting the site at that moment or not. Scale-to-Zero is a different operating mode: if nobody connects for a certain period of time, the program is completely shut down and the rented computer is released. The number of running copies thus drops to zero, hence the name. If a request comes in afterward, the program is automatically restarted and answers it. For the operator, this means: no usage, no bill.
Why idle time is so expensive
Most programs are used extremely unevenly. A learning app has a lot of traffic in the afternoon and almost none at night. An internal company tool sits completely idle over the weekend. Anyone paying by the hour in the classic way still pays continuously for machines that aren’t doing anything. Calculated over a year, a large part of the bill then turns out to be pure idle time.
This becomes especially clear with AI applications. They often run on graphics cards, so-called GPUs, which specialize in massive numerical computation. Renting such a card can easily cost several euros per hour, meaning an amount in the four-figure range per month. If a prototype only uses it for a few minutes a day, the ratio is absurd. Scale-to-Zero is precisely what makes such projects affordable in the first place.
However, there is also a downside worth knowing about. Anyone with users accessing the service around the clock saves nothing at all through Scale-to-Zero. There, the constant shutting down and starting up only creates additional disruption. So the technology is worthwhile mainly for heavily fluctuating or infrequent usage.
The cold start and what it costs
For the shutdown to work, the platform monitors incoming traffic. If nothing happens for a defined period of time, say five minutes, the program is stopped. However, a small routing hub stays awake and keeps listening for requests. If one comes in, it briefly holds it, restarts the program, and then forwards the request. The user only notices that it takes a bit longer.
This delay is called a cold start. It occurs because the program, along with all required files, has to be loaded again from scratch. For a lightweight web application, this is often under two seconds. For an AI model, it can take considerably longer, because several gigabytes of learned numerical values have to be copied into the graphics card’s memory. Delays of half a minute are not uncommon here.
Providers use various tricks to counter this. Some keep the model data ready in a fast cache located directly next to the machine. Others offer to always keep at least one copy warm. Strictly speaking, that is no longer Scale-to-Zero, but rather a compromise between cost and speed.
Where the principle shows up in everyday life
Scale-to-Zero is best known under the keyword serverless. Services like AWS Lambda, Google Cloud Run, or Cloudflare Workers only bill for the milliseconds actually consumed. Many platforms for AI models, such as Replicate or Hugging Face Spaces, also automatically shut down unused applications. If a small AI demo online takes forever to load on the first click and then runs smoothly afterward, you’ve just experienced a cold start.
The term comes up in financial and tech news when the topic is cloud costs. Companies then report having cut their spending on test environments by large percentages. For investors, this is an indicator of just how closely cloud providers now tie their pricing models to actual usage.
