We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningIntermediate · API Design & Protocols
Controlling how many requests a client can make in a given time window. Protects your API from abuse and ensures fair usage.
In short
Rate limiting caps how many requests a client can make in a window of time, so one user, a buggy script, or an attacker cannot exhaust your capacity and degrade service for everyone else. The most common algorithm is the token bucket, which allows short bursts while holding the long-run average to a fixed rate.
In short
Rate limiting caps how many requests a client can make in a window of time, so one user, a buggy script, or an attacker cannot exhaust your capacity and degrade service for everyone else. The most common algorithm is the token bucket, which allows short bursts while holding the long-run average to a fixed rate.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Rate Limiting across a run of lessons, not one page. These 4 alone are about 115 minutes of step-by-step reading, 1 of them with code you run in the browser, all with a quiz.
Protect your API from abuse and overload by controlling how many requests each consumer can make.
Putting it all together, designing rate limiting for production APIs at scale.
Design a distributed rate limiting system - token bucket, sliding window, and protecting services at massive scale.
A bucket of tokens that refills at a steady rate, the most popular rate limiting algorithm in production.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Rate Limitingpay once,
yours for life
Every service has a ceiling on how much work it can do per second. Rate limiting is how you stop a single source from eating that whole ceiling. Without it, one client looping on a bug, a scraper pulling your catalog, a credential-stuffing attack, or a sudden viral spike can drive load past what the system can handle, and everyone's requests slow down or fail together.
It is also about fairness and cost. On a shared API, you do not want one heavy user starving the rest. On a paid API, the limit is how you enforce plan tiers. And in front of an expensive downstream, like a payment processor or a third-party API you are billed for, the limiter protects your bill as much as your servers.
The token bucket is the workhorse. Picture a bucket that holds up to a fixed number of tokens, the capacity. Tokens are added at a steady refill rate, say ten per second, and the bucket never holds more than its capacity. Every request must remove one token to proceed. If a token is available, the request is allowed and one token is spent. If the bucket is empty, the request is rejected, usually with HTTP status 429 Too Many Requests.
The clever part is the capacity. Because unused tokens accumulate up to that limit, a client that has been idle can briefly burst above the steady rate, spending the saved tokens, and then settle back to the refill rate once they run out. That matches how real traffic behaves, bursty but bounded, which is why the token bucket feels fairer than a hard fixed cap.
Two numbers define the policy: the refill rate sets the long-run average requests per second, and the capacity sets how large a burst you tolerate. Tune them separately. A high capacity with a modest refill rate allows spiky clients; a capacity equal to the refill rate behaves almost like a smooth, even limit.
The token bucket
Tokens refill at a fixed rate up to a capacity. Each request needs one token. Spare tokens absorb short bursts; an empty bucket rejects.
Leaky bucket is the close cousin: requests enter a queue that drains at a fixed rate, which smooths bursts out completely rather than allowing them. It is good when you need a strictly even outflow, for example protecting a downstream that cannot handle spikes at all.
Fixed window counting is the simplest: count requests per clock minute and reset at the boundary. It is easy but has an edge case, a client can send a full window's worth of requests at the end of one window and again at the start of the next, briefly doubling the intended rate. Sliding window log and sliding window counter fix that by tracking requests across a rolling interval, at the cost of more memory or a little approximation.
For most APIs the token bucket is the default choice because it is cheap to compute, needs only two numbers per client (token count and last refill time), and handles bursts gracefully.
A limiter is only correct if the count is shared. If you run ten API gateway instances and each keeps its own in-memory bucket, a client spread across all ten effectively gets ten times the limit. So production limiters keep the counter in a fast shared store, almost always Redis, and update it atomically.
A common pattern is a small Redis script that, in one atomic step, refills the bucket based on elapsed time, checks whether a token is available, decrements if so, and returns allow or deny. Doing it in one atomic operation avoids race conditions between gateway instances hitting the same client's bucket at once. Limits are typically keyed by API key, user id, or client IP, and you often run several limits at once, for example per-second and per-day.
In a real system
The limiter usually lives at the edge or the API gateway, with a shared counter store like Redis so the limit holds across many gateway instances.
Cloudflare
Runs rate limiting at the edge across its global network, stopping abusive traffic before it ever reaches your origin.
Stripe
Publishes per-endpoint request limits and returns 429 with headers telling you how long to wait, a textbook token-bucket-style API limiter.
NGINX
Its limit_req module implements a leaky-bucket limiter you can put in front of any upstream with a few lines of config.
Redis
The usual shared counter store behind distributed limiters, with atomic scripts that refill and check a client's bucket in one step.
Rate limiting restricts how many requests a client can make to a service within a time window. It protects the service from overload and abuse, enforces fair use among clients, and is how paid APIs enforce plan tiers. Requests over the limit are typically rejected with HTTP status 429.
A bucket holds up to a fixed number of tokens (the capacity) and refills at a steady rate. Each request removes one token. If a token is available the request is allowed; if the bucket is empty it is rejected. Because unused tokens build up to the capacity, a previously idle client can briefly burst above the steady rate.
Token bucket allows bursts: saved-up tokens let a client temporarily exceed the steady rate. Leaky bucket forces a strictly even output by draining a request queue at a fixed rate, smoothing bursts out entirely. Use token bucket when short bursts are acceptable, leaky bucket when the downstream needs a perfectly even flow.
HTTP 429 Too Many Requests. Well-behaved APIs also return a Retry-After header, and often headers showing the limit, how many requests remain, and when the window resets, so clients can back off correctly.
Keep the counter in a shared, fast store such as Redis rather than in each server's memory, and update it atomically (commonly with a small Lua script that refills and checks the bucket in one step). Otherwise a client spread across N servers effectively gets N times the intended limit.
API Rate Limiting
Putting it all together, designing rate limiting for production APIs at scale
Design a Rate Limiter
Design a distributed rate limiting system - token bucket, sliding window, and protecting services at massive scale
Token Bucket Algorithm
A bucket of tokens that refills at a steady rate, the most popular rate limiting algorithm in production
API Gateway Security
Centralizing authentication, rate limiting, and threat protection at the API gateway layer
Leaky Bucket Algorithm
Requests drip out at a constant rate, smooth traffic shaping for APIs and networks
Throttling
Slowing down the rate of processing requests instead of rejecting them outright. The gentler cousin of rate limiting.
API Gateway
A single entry point for all client requests that routes them to the appropriate microservice. Handles auth, rate limiting, and request transformation.
Redis
An in-memory data store used as a cache, message broker, and database. Blazing fast because everything lives in RAM.
WebSocket
A protocol for full-duplex communication over a single TCP connection. Unlike HTTP, the server can push data to the client without being asked.
SSE
Server-Sent Events: a one-way channel where the server pushes updates to the client over HTTP. Simpler than WebSockets when you only need server-to-client streaming.
GraphQL
A query language for APIs where the client specifies exactly what data it needs. No over-fetching, no under-fetching. One endpoint to rule them all.
This page explains the idea. The "Rate Limiting" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Rate Limiting lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Rate Limiting in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
