p75 Latency

When you submit a search request, time passes until the response appears. This wait time is called latency. It is never the same for every request: sometimes the response arrives in 200 milliseconds, sometimes it takes two seconds. To combine these many individual values into a single number, they are sorted by size. The p75 latency is then the value below which 75 percent of all requests fall. In other words: three out of four requests are at least this fast, while a quarter are slower.

Why the average is misleading here

The obvious approach would be to simply calculate the mean of all wait times. But that is exactly what often leads astray. Latencies are unevenly distributed: most requests are fast, while a few take extremely long. A single outlier of 30 seconds can pull the mean sharply upward, even though almost all users had a good experience.

Conversely, a good average can hide a real problem. Suppose the average is 300 milliseconds. That sounds good. But if every fifth request takes four seconds, a large portion of users are still frustrated. Percentiles make such cases visible because they ask about the distribution rather than the mean.

p75 is a deliberately chosen compromise. The value describes the experience of the clear majority, but doesn’t react immediately to every single glitch. That’s why companies like to use p75 as a target for normal operation. For the worst cases, one additionally looks at p95 or p99, i.e. the slowest twentieth or hundredth.

How the number comes about

A server records for each request how long it took. Over a time window, say five minutes, this accumulates thousands of measured values. These values are sorted, and the value at the 75 percent position is read off. With 1000 sorted requests, that is the 750th one — its value is the p75 latency for this window.

In practice, nobody stores all individual values permanently, since there are simply too many. Instead, measurement systems maintain so-called histograms: they only count how many requests fell into time spans like 0–100 ms, 100–200 ms, and so on. The percentile is then estimated from these buckets. The number is therefore slightly imprecise, but good enough for practical purposes.

A common misconception: percentiles cannot simply be averaged together. The mean of the p75 values from ten servers is not the p75 of the entire system. Anyone who wants the overall picture must merge the raw data or the histograms and recalculate the percentile.

p75 in contracts, dashboards, and AI services

Anyone buying software as a service often finds percentiles in the contract. It might state, for instance, that the p75 latency must stay below 500 milliseconds. Such commitments are called Service Level Agreements. An average would be a weaker promise for the customer, because it can hide outliers.

Google also evaluates websites using percentiles, among other things in the so-called Core Web Vitals. What matters there is not how quickly the page loads on the developer’s machine, but how fast it is for real users with a weak phone and poor network. A better p75 can thus directly influence search ranking.

The value also appears with AI providers. A chatbot uses the finished model to compute answers — this step is called inference. How long it takes depends on the length of the request and on the load of the data centers. Companies therefore report how many milliseconds pass until the first word, stating both p75 and p99 for it. If p75 rises over days, that is a signal of insufficient computing capacity.