We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
6 system design questions lean on this idea. Each walks through the full answer.
System Design Masterclass · Reliability Resilience
Free preview: the first 2 parts of this 7-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Reliability Resilience
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
Stripe allows 100 requests per second per API key. GitHub allows 5,000 requests per hour per authenticated user. Twitter allows 300 tweets per 3 hours. Shopify allows 40 requests per second per app.
These limits exist because without them, a single buggy client can take down the service for everyone. In January 2023, a partner integration at a major payment processor had a bug that sent the same API call 200,000 times in under a minute. Without rate limiting, those 200,000 calls would have consumed every available thread in the , starving thousands of other merchants of the ability to process payments. With rate limiting, the integration received 429 responses after its first 100 calls, and every other merchant was unaffected.
is the first line of defense for any service that accepts traffic from the outside world. It caps how much work any single caller can create. Unlike load shedding (which protects aggregate capacity) and backpressure (which propagates slowness through pipelines), rate limiting answers a simpler question: is this specific caller sending too much traffic?
The four dominant algorithms for answering that question are fixed window, sliding window, token bucket, and leaky bucket. Each makes a different trade-off between burst tolerance, memory cost, and implementation complexity. By the end of this lesson you will know which algorithm fits which use case and how to enforce a global limit across hundreds of edge servers.
Every answers the same question: has this caller exceeded their quota? The algorithms differ in how they count and what happens at the boundary.
| Algorithm | How It Counts | Burst Behavior | Memory Cost | Used By |
|---|---|---|---|---|
| Fixed window | Counter resets every N seconds | Allows 2x burst at window boundary | 1 counter per key | Simple internal tools |
| Sliding window log | Stores timestamp of every request | No boundary burst, exact | O(requests) per key | Low-volume APIs |
| Sliding window counter | Weighted average of current and previous window | Small boundary burst | 2 counters per key | Cloudflare, most CDNs |
| Token bucket | Tokens refill at a steady rate, burst up to bucket size | Allows controlled bursts | 2 values per key | Stripe, GitHub, AWS |
| Leaky bucket | Requests enter a queue that drains at a fixed rate | No bursts, constant output | 1 queue per key | Cloudflare Workers (internal), video transcoders |
Fixed window is the simplest: a counter that resets to zero at the start of every window. The problem is that a caller can send their full quota at the end of one window and the full quota at the start of the next, creating a 2x burst in a short span. Sliding window counter fixes this by weighting the previous window's count into the current window's calculation.
Token bucket is the most popular for public APIs because it explicitly allows bursts. Stripe documents 100 requests per second with a burst capacity of 25. That means a client that has been idle for a few seconds can fire 25 requests instantly, then settle into 100 per second. Interactive API clients love this because their first page load often needs to make several parallel requests.