We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningFoundation · Core Fundamentals
A single entry point for all client requests that routes them to the appropriate microservice. Handles auth, rate limiting, and request transformation.
In short
An API gateway is a server that sits in front of your backend services and acts as the single entry point for every client request. It receives the incoming call, handles cross-cutting work like authentication, rate limiting, and request routing, then forwards the request to the correct service and returns the response to the client.
In short
An API gateway is a server that sits in front of your backend services and acts as the single entry point for every client request. It receives the incoming call, handles cross-cutting work like authentication, rate limiting, and request routing, then forwards the request to the correct service and returns the response to the client.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers API Gateway across a run of lessons, not one page. These 4 alone are about 132 minutes of step-by-step reading, every one with a quiz.
Every mechanism in this chapter running in one component in front of everything, what that concentration costs, and how to keep it from becoming an application.
Design an API gateway - request routing, authentication, rate limiting, circuit breaker, load balancing, and observability.
Routing on the one field the user cannot see, who is allowed to write it, and what it does to every cache in front of you.
Centralizing authentication, rate limiting, and threat protection at the API gateway layer.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with API Gatewaypay once,
yours for life
In a microservices system you might have 30 or 40 services, each owning a slice of the product: users, orders, payments, search, notifications. If every mobile app and browser had to know the address of each service, talk to each one directly, and re-implement login and rate limiting for all of them, the client would be a mess and every service would duplicate the same plumbing.
An API gateway fixes this by being one address that all clients call. The client sends a request to api.example.com, and the gateway decides which internal service should handle it. To the outside world there is one API. Behind the gateway there can be dozens of services on private network addresses that clients never see.
It is best to think of the gateway as a reverse proxy with extra responsibilities. A plain reverse proxy just forwards traffic. A gateway also checks who you are, whether you are allowed to make this call, how often you are allowed to make it, and sometimes reshapes the request and response along the way.
When a request arrives, the gateway runs it through an ordered chain of steps before any backend sees it. First it terminates TLS so the rest of the traffic inside your network can stay plain HTTP. Then it authenticates the caller, usually by validating a JWT or an API key, and rejects anything that fails with a 401 before it ever reaches a service.
Next it applies rate limiting and quotas, often backed by Redis counters, so one noisy client cannot flood your backend. Then it matches the request path and method against a route table. A rule like POST /orders forwards to the order service, while GET /search/* forwards to the search service. The gateway rewrites the path if needed and adds headers such as the resolved user id so the downstream service does not have to re-verify the token.
Some gateways also do request aggregation. A single call from a mobile home screen might fan out to the user, orders, and recommendations services in parallel, then the gateway stitches the three responses into one payload. This pattern, often called backend for frontend, cuts the number of round trips a slow mobile connection has to make.
Finally the gateway records metrics, latency, and status codes for every call, which gives you one place to see the health of the whole API instead of scraping logs from 40 services.
Reach for a gateway once you have more than a handful of services and more than one type of client. The payoff is real: auth, rate limiting, TLS, and logging live in one place instead of being copied into every service, and you can change a routing rule or block an abusive client without redeploying anything downstream.
The cost is that the gateway is now a single point of failure and a potential bottleneck. If it goes down, every API call goes down with it, so you run it as multiple replicas behind a load balancer and keep its logic thin. Every request also pays a small extra hop in latency, usually a few milliseconds, which is almost always worth it but is not free.
The other risk is the fat gateway. It is tempting to push business logic into the gateway because it is a convenient choke point. Resist that. Keep it focused on routing, security, and traffic shaping. Business rules belong in the services. A gateway stuffed with domain logic becomes a deployment bottleneck that every team has to coordinate around.
Picture a food delivery app. The phone makes one request to api.delivery.com/checkout with a bearer token. The gateway terminates TLS, validates the token against the auth service, and confirms the user has not exceeded 100 requests per minute.
It then routes the call to the order service, which creates the order, and the gateway also calls the payment service to authorize the card. If payment fails the gateway returns a clean 402 to the phone. The phone never learned the addresses of the order or payment services, never handled token verification itself, and made a single network call instead of three.
If the company later splits the order service into separate cart and order-history services, only the gateway route table changes. The mobile app keeps calling the same /checkout URL and never notices the refactor.
The same front-door idea now sits in front of AI model providers. The LLM gateway page covers virtual keys, budgets, fallback between providers, and why a gateway that holds every provider key has to be guarded like a vault.
Netflix Zuul
Netflix built and open sourced Zuul as the front door for its streaming API, handling routing, auth, and dynamic filtering for billions of requests a day.
Amazon API Gateway
AWS managed gateway that fronts Lambda functions and HTTP services, handling auth, throttling, and request mapping without you running any servers.
Kong
Popular open source gateway built on Nginx, used by companies to add auth, rate limiting, and observability to existing APIs through plugins.
Kubernetes Ingress and Gateway API
In Kubernetes clusters, ingress controllers and the newer Gateway API route external traffic to the right internal service and terminate TLS at the cluster edge.
A load balancer spreads traffic across identical copies of one service to share load. An API gateway routes traffic to different services based on the request path and also handles auth, rate limiting, and request shaping. They often sit together: a load balancer in front of multiple gateway replicas.
A gateway is a reverse proxy with extra jobs. Both forward client requests to backend servers, but a gateway adds authentication, rate limiting, request aggregation, and per-API routing rules, while a plain reverse proxy mainly forwards and caches traffic.
Yes, it adds one extra network hop, usually a few milliseconds. That cost is almost always worth it because the gateway removes duplicated auth and rate limiting work from every service and gives you one place to observe and control traffic.
Usually not. With one or two services and a single client, a gateway is overhead you do not need yet. It earns its place once you have several services, multiple client types, or cross-cutting concerns like auth and throttling that you do not want to copy everywhere.
Every request through it fails, since it is the single entry point. That is why production gateways run as multiple stateless replicas behind a load balancer across availability zones, and keep their logic thin so they restart fast and rarely crash.
Design an API Gateway
Design an API gateway - request routing, authentication, rate limiting, circuit breaker, load balancing, and observability
Header-Based Routing
Routing on the one field the user cannot see, who is allowed to write it, and what it does to every cache in front of you
API Gateway Security
Centralizing authentication, rate limiting, and threat protection at the API gateway layer
Load Balancer
Distributes incoming traffic across multiple servers so no single server gets overwhelmed. Like a traffic cop directing cars to different lanes.
Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Rate Limiting
Controlling how many requests a client can make in a given time window. Protects your API from abuse and ensures fair usage.
Latency
The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.
Throughput
The number of operations a system can handle per unit of time. Think of it as how many cars a highway can move per hour.
Bandwidth
The maximum amount of data that can be transferred over a network in a given time. It's the width of the pipe, not how fast the water flows.
This page explains the idea. The "API Gateway" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the API Gateway lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about API Gateway in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
