We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
This idea carries a full system design question on its own. Each walks through the full answer.
System Design Masterclass · Api Design Protocols
Free preview: the first 2 parts of this 7-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Api Design Protocols
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
You know the algorithms: token bucket, leaky bucket, sliding window. Now the question is: where do you put them? How do you handle distributed deployments? What happens at the vs. the application layer? How do you configure different limits for different API tiers?
This lesson is about the architecture of in production, not the algorithm. Stripe, GitHub, and Shopify all publish their rate limiting designs. The patterns are consistent: multiple layers, shared counters, and standardized response headers.
Layer 1: / Edge (Cloudflare, AWS CloudFront) First line of defense. Blocks DDoS and bot traffic before it reaches your servers. Configured per-IP or per-region. Coarse-grained but fast. Cloudflare's edge network absorbs 10 Gbps+ volumetric attacks without any request touching your origin. Limits here are crude: "no single IP can exceed 500 req/s on /login."
Layer 2: (Kong, AWS API Gateway, Envoy) Per-API-key . Checks the key, looks up the tier, applies the limit. This is where most "official" rate limits are enforced. Returns 429 with standard headers. The gateway uses Redis as a shared counter so that all gateway pods agree on the same count. Without a shared counter, horizontally scaled gateways leak past the limit.
Layer 3: Application (middleware) Per-user or per-endpoint rate limiting with business context. The login endpoint might allow 5 attempts per minute. The search endpoint might allow 100 per minute. The bulk export endpoint might allow 1 per hour. Only the application knows what a "project" or "export" costs. The gateway sees URLs and keys, not business semantics.
Layer 4: Database / Backend
Connection limits and query rate limits. PgBouncer limits connections to Postgres. has maxclients. Postgres itself enforces max_connections (typically 100-300). These are not API rate limits but they are the ultimate backstop. If every upstream layer fails, the database refuses new connections instead of crashing.