We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Api Design Protocols
Free preview: the first 2 parts of this 7-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Api Design Protocols
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
is binary: you're either under the limit (request accepted) or over it (request rejected with 429). But what if instead of slamming the door, you could slow the door, processing requests at a reduced rate?
That's throttling. While rate limiting rejects excess requests, throttling slows them down. Instead of returning 429 immediately, the server delays the response, queues the request, or processes it with reduced priority. The client still gets a response, just slower.
Think of a highway on-ramp. During rush hour, the traffic light at the ramp turns green for one car at a time, metering traffic onto the highway at a controlled rate. Cars aren't turned away (rejected). They're just slowed down (throttled). The highway keeps flowing, at reduced speed but without gridlock.
Throttling is the intermediate step between "everything's fine" and "service unavailable." It preserves availability at the cost of , a tradeoff that's usually worth making.
| Throttling | ||
|---|---|---|
| Response to excess | Reject immediately (429) | Slow down or degrade |
| Client experience | Immediate error | Delayed but successful response |
| Server load | Drops excess load | Manages load smoothly |
| Use case | Protect from abuse | Graceful degradation under pressure |
Throttling strategies:
Response delay. Add artificial to excess requests. Process them, but slowly. A client sending 200 req/sec when the limit is 100 gets responses, but each takes 500ms instead of 50ms.
Queue-based throttling. Excess requests are queued and processed at a controlled rate. The client waits longer, but gets a response. If the queue is full, then requests are rejected.
Priority-based throttling. Under load, serve premium customers at full speed and throttle free-tier users. This is how API tiers work in practice, not by rejecting, but by deprioritizing.
Feature degradation. Instead of slowing the whole response, disable expensive features. Search results skip personalization. Product pages serve cached data instead of real-time inventory. The API still responds quickly, but with reduced functionality.