Percentiles

Amazon doesn't optimize for average response time. They optimize for the P99.9, the slowest 0.1% of requests. Why? Because their highest-value customers, the ones with the most purchase history, the fullest carts, the most active wishlists, tend to have the slowest page loads. Their accounts have the most data to fetch, the most recommendations to compute, the most personalization to apply.

If Amazon only looked at the average, those high-value customers would have a terrible experience while the average stayed "fine." The P99.9 is where the money is.

Percentiles tell you what percentage of your users experience performance at or below a given threshold. P50 is the median, half of users are faster, half are slower. P95 means 95% of users are at or below that value. P99 means only 1% of users are slower than that number. Each step toward the tail tells you about an increasingly frustrated minority of your user base.

If you measured 1,000 API responses and sorted them from fastest to slowest:

The gap between P50 and P99 tells you how consistent your service is. If P50 is 45ms and P99 is 100ms, the experience is fairly uniform. If P50 is 45ms and P99 is 3 seconds, something is making a small percentage of requests dramatically slower, cache misses, cold starts, garbage collection pauses, or heavy queries.

Which percentile should you monitor? At minimum: P50 (the typical experience), P95 (where most users land), and P99 (the tail). For high-traffic services: add P99.9. The right choice depends on traffic volume, at 10 million requests/day, P99.9 represents 10,000 slow requests. That's a lot of frustrated users.