Why Averages Lie in Capacity Planning

The average is the most seductive number in infrastructure. It is one value, it is stable, it fits in a dashboard panel, and it goes up and to the right or down and to the left in ways executives can read at a glance. It is also, in capacity planning, the number most likely to be quietly wrong. Not wrong in the sense of miscalculated. Wrong in the sense that the calculation is correct and the conclusion it invites is not.

Capacity is a tail problem

Systems do not fail at their average load. They fail at their peaks. A service averaging 40 percent CPU can be throttling hard for the five minutes a day that actually matter, and the daily mean will never tell you. The question capacity planning answers is not "what does this workload usually need" but "what does this workload need when it would otherwise fall over." That is a high percentile or a max over a window, never a mean. The first lie of the average is omission: it summarizes the 95 percent of time when nothing interesting is happening.

Percentiles do not average

The second lie is subtler and more common in fleet-level reporting. Someone computes a P95 per node, or per pod, or per cluster, and then averages those P95s to get a "fleet P95." That number has no statistical meaning. Percentiles are not linear. The mean of per-entity P95s is not the P95 of the combined population, and depending on how load is distributed it can be wildly above or below it. If you need a fleet-level percentile, compute it from the merged distribution. If you need a summary of per-entity percentiles, say that explicitly and pick the summary deliberately: a max if one bad entity should dominate the decision, a high percentile if you want to drop pathological outliers. "Average of P95s" is almost never the statistic anyone actually wanted.

Unequal populations break naive aggregation

The third lie is weighting. Average the utilization of a ten-node cluster and a six-hundred-node cluster and each contributes equally to the result. The small cluster, which barely matters to the bill or the risk, moves the fleet number as much as the one that dominates both. Efficiency ratios make this worse: averaging ratios across entities of different sizes produces a figure that corresponds to no real population. The same trap appears one level down when pods of very different sizes are averaged inside a service, and one level up when teams or regions are compared on their "average efficiency."

The rule: aggregate per entity first

The discipline that fixes all three failures is the same. Decide what decision the number serves, then compute the statistic at the level where the decision is made, and only then roll up. Within an entity, summarize with the statistic the failure mode demands, usually a high percentile or a max over an honest window. Across entities, choose the aggregation that matches the question: a max when one entity failing is the whole story, a percentile across entities when you want robustness against outliers, a properly weighted total when you are reasoning about money. Never let a mean hop across a population boundary on its own.

A checklist

Before trusting any aggregate in a capacity decision, I ask four questions. What decision does this number feed? Is the failure mode a tail or a total? Did any percentile get averaged anywhere in the pipeline that produced this number? And are the entities being combined remotely comparable in size? If a dashboard cannot answer those four questions, the number on it is decoration. The work of performance engineering is not collecting metrics. It is refusing to let convenient numbers make inconvenient decisions.