Blog
System design, written down properly
Walkthroughs, interview problems and engineering notes. Every post is built from something that was actually measured or built, not summarised from elsewhere.
Roni Das
Lessons and tutorials at systemdesign.academy
106 posts · page 1 of 6
BigBasket System Design: Delivery Slots, 10-Minute Orders and Items Sold by Weight
BigBasket sells groceries online in India. It is owned by Tata Digital since May 2021. It is known for big monthly baskets delivered in a time slot the customer picks. It also runs BB Now, a 10-minute service from dark stores. A design like BigBasket's must handle both. A slot order is large and planned. It is packed in a big warehouse and carried by van with many other orders. A 10-minute order is small and urgent. It is packed in a dark store near the customer. Three things make this design its own. First, a delivery slot is stock: each slot holds only so many orders. Second, one cart may have to become two orders. Third, fruit and vegetables are sold by weight, so the final bill is known only after packing.
readDesign a Hit Counter: Last 5 Minutes, Ring Buffer and View Counts at Scale
A hit counter records events and answers one question: how many happened recently. The classic version is LeetCode 362. It counts hits in the past 300 seconds. The best answer keeps 300 small buckets, one per second, and reuses them in a circle. So memory stays fixed at about 4.8 KB, however busy the site is. The follow-up turns it into a real system. Many web servers see the hits. A shared counter in one key gets hammered. So each server adds up its own hits for one second, then sends one total. Hot keys are split into sub-keys. The last-five-minutes view comes from time buckets. Unique viewers come from a HyperLogLog, a small sketch that estimates how many different people came.
readLenskart System Design: Prescriptions, Lens Labs and Stores as One System
Lenskart sells glasses, sunglasses and contact lenses in India and abroad. It sells online and in its own stores. A design like Lenskart's has three parts that a normal shop does not. First, the prescription. It is medical data with a strict format, and a wrong number means glasses that give headaches. Second, the lens. Common powers are kept in stock. Unusual powers and special lenses must be made in a lab. So each order must decide which path it takes. Third, the order is a job with many steps. Pick the frame, get the lenses, cut, fit, check, pack and ship. Every step can fail and must be tracked. On top sits omnichannel. A customer may get an eye test in a store, then buy online, or try frames at home first.
readMyntra System Design: Fashion E-Commerce and the End of Reason Sale
Myntra sells fashion online in India. Flipkart bought it in May 2014. Its big event is the End of Reason Sale, which runs twice a year. Fashion changes the usual shop design in four ways. First, stock is per size. A shirt in six sizes is six separate counts, and the popular sizes run out first. Second, prices change at a fixed minute when the sale starts. Millions of people are waiting for that minute. Third, stock sits in warehouses and in brand stores, so each order must pick where to ship from. Fourth, many fashion items come back. So returns are a normal path, with pickup, a check, a refund and a restock.
readPolicybazaar System Design: Insurance Quotes from Many Insurers at Once
Policybazaar is an insurance marketplace in India. Since June 2021 it has held an insurance broking licence from the regulator, IRDAI. A customer enters details like age, city and the cover they want. The site asks many insurers for a price and shows the plans side by side. Then the customer picks one, fills a proposal form, pays, and gets a policy. Every year, the policy comes up for renewal. Three parts are hard. First, the quote fan-out: many slow outside systems, each with its own rules, must answer within a few seconds. Second, buying is a hand-off. The payment is taken by one system and the policy is issued by another company, the insurer. Third, the data is sensitive. Health details and ID numbers need strong protection.
readDesign a Video Conferencing System: WebRTC, SFU vs MCU, Simulcast and Cascading
A video conferencing system lets a group of people send live audio and video to each other with very little delay. First, some terms. WebRTC (Web Real-Time Communication) is the open standard built into every modern browser and phone for live audio, video and data. A track is one stream of media, such as one camera or one microphone. Latency is delay; jitter is how much that delay changes from packet to packet. A NAT (network address translation) is the box, usually your home or office router, that lets many devices share one public IP address; it also blocks most traffic that you did not start yourself. The design has two planes. The signaling plane is a small, reliable channel, usually a WebSocket (a connection that stays open so the server can send messages at any time). It handles joining a room, sign-in, and swapping SDP (Session Description Protocol), the text that says which codecs and tracks each side will use. The media plane carries the actual audio and video as SRTP (Secure Real-time Transport Protocol, encrypted media packets), usually over UDP, because a late packet of live audio is useless and UDP does not stop to resend it. To get through NATs, each client uses ICE (Interactive Connectivity Establishment): it collects possible addresses with the help of a STUN server, tests them, and falls back to a TURN server that relays its media when no direct path works. In the middle sits a media server, almost always an SFU (Selective Forwarding Unit): every client uploads its tracks once, and the SFU forwards each viewer the copies it needs without decoding them. Simulcast and SVC let the SFU give each viewer a quality that fits their screen and network, and congestion control decides that quality from moment to moment. For a global product, SFUs in several regions are joined into one room (cascading), and recording and large webinars are built as extra subscribers to the same SFUs.
readDesign an Authentication System: Passwords, Sessions, JWT, Refresh Tokens and MFA
An authentication system proves who a user is. It is different from authorization, which decides what that user is allowed to do. First, some terms. A credential is the proof a user shows: a password, a code from a phone, or a passkey. A password hash is the result of running the password through a slow one-way function, so the database never stores the password itself. A salt is a random value stored with each hash, so two users with the same password get different hashes. After the user proves who they are, the server gives them something to show on later requests, so they do not type the password every time. That can be a session: a long random id kept in a cookie, with the details stored on the server. Or it can be a token: a signed piece of data, such as a JWT (JSON Web Token), that any service can check with a public key without asking the database. Most apps use a short-lived access token (about 15 minutes) plus a long-lived refresh token that can get a new access token. Multi-factor authentication (MFA) asks for a second proof, such as a 6-digit one-time password (OTP) from an app or an SMS. The design must also slow down attackers who guess passwords, give a safe way to reset a forgotten password, and let a user log out of a lost phone.
readDesign an Email Service like Gmail: SMTP, Mailbox Storage, Search and Spam Filtering
An email service receives, stores, sends and searches mail for its users. First, some terms. SMTP (Simple Mail Transfer Protocol) is how mail servers hand a message to each other. An MX record is a DNS entry that says which server accepts mail for a domain, so a sender looks up the MX record of gmail.com to know where to deliver. An MTA (mail transfer agent) is a server that sends and receives mail over SMTP. A mailbox is one user's stored mail, organised into folders or labels. Users read mail through the web app, a mobile app, or older protocols such as IMAP, which let any mail app sync a mailbox. The design splits each message into two parts. The small metadata (sender, subject, date, labels, read or unread) goes in a database, indexed for fast listing. The large body and attachments go in a blob store, a store for large files such as object storage. A per-user search index lets users find old mail. Spam filtering decides which mail reaches the inbox. Three sender checks help it. SPF asks: is this server allowed to send for that domain? DKIM asks: is the message signed by that domain? DMARC says what to do when those checks fail. Outgoing mail waits in a queue and is retried when the other side is busy.
readDesign an Order Matching Engine: Order Book, Price-Time Priority and Order Types
An order matching engine pairs buy orders with sell orders. First, some terms. An order says: buy or sell, how many (the quantity), and usually a price limit. A limit order says the worst price you accept, for example 'buy 100 at 101 or less'. A market order takes the best prices available now. The order book is the list of limit orders that are still waiting, called resting orders. Buy orders are bids and sell orders are asks. Each side is grouped into price levels, and each level is a queue of orders at that price in the order they arrived. The bid side is sorted from the highest price down, and the ask side from the lowest price up. The best bid and best ask are the top of the book. When a new order arrives, the engine checks if it crosses: a buy crosses when its limit is at or above the best ask. If so, it trades against the best price first, and at each price against the oldest order first. This rule is called price-time priority, or FIFO (first in, first out). The trade happens at the resting order's price. Whatever is left of a limit order then rests in the book; what is left of a market or IOC order is dropped. Time in force rules say how long an order lives: GTC (good till cancelled), IOC (immediate or cancel: fill what you can now, drop the rest) and FOK (fill or kill: fill all of it now or nothing). The engine also stops a trader from trading with themselves (self-trade prevention). Some futures markets split a fill across the orders at a price in proportion to their size instead (pro rata). To be fast and fair, each book is run by a single thread in memory, fed by one ordered stream of input that is written to a journal first. Same input in the same order gives the same trades, so a replica can replay the journal and hold an identical book.
readDesign a Distributed Logging System: Agents, Kafka, Indexing, Retention and Search
A distributed logging system collects log lines from many servers, stores them centrally, and makes them searchable. First, some terms. A log line is one record a program writes, such as an error with a time and a message. Structured logs are written as fields (JSON is common), for example level=ERROR, service=checkout, trace_id=abc, instead of free text, so machines can filter them. An agent is a small program on each server, such as Fluent Bit, Vector or the OpenTelemetry Collector, that reads the logs and ships them. Kafka is a system that stores a stream of messages on disk, in order, so other programs can read them later. Object storage, such as Amazon S3, is cheap storage for whole files. The usual design has five stages. Collect: apps write to standard output or a file, and an agent on each machine reads it. It adds labels such as the service and the pod (one running copy of an app in Kubernetes), and buffers on disk if the next stage is slow. Buffer: a durable queue, usually Kafka, takes everything, so a slow or broken store never pushes back on the apps. Process: consumers (programs that read from Kafka) parse lines, remove secrets and personal data, drop or sample noise, and route by team. Store: a hot tier keeps recent logs ready to search. It is one of three kinds: a full-text index of every word (Elasticsearch or OpenSearch), a small index of labels with the text in compressed chunks (Grafana Loki), or a column store (ClickHouse). Then a cheap cold tier in object storage keeps logs for months or years. Query: a UI such as Grafana or Kibana, plus alerts on error patterns. With 5,000 servers writing 100 lines a second of 500 bytes each, that is 250 MB a second, 21.6 TB a day before compression, which is why indexing choices and retention decide the cost.
readDesign a Load Balancer: L4 vs L7, Algorithms, Health Checks and High Availability
A load balancer spreads incoming requests across a group of servers, called a pool or target group, so that no one server is overloaded and a broken server gets no traffic. First, some terms. A request is one call from a user, such as loading a page. A connection is the line a client opens to a server to send requests over; one connection can carry many requests. TCP is the internet's standard way to carry a connection; TLS is the encryption on top of it, the S in HTTPS. There are two main kinds of load balancer. A layer 4 (L4) load balancer sees only network addresses and ports (numbers that say which program on a machine gets the data). It picks a server per connection, very fast, and works for any TCP or UDP traffic (UDP is a simpler way to send data that does not wait for proof it arrived). AWS says its Network Load Balancer works at layer 4 and can handle millions of requests per second. A layer 7 (L7) load balancer, also called a reverse proxy (a server that stands in front of other servers and passes requests to them), ends the TLS connection, reads the HTTP request, and can route by path or header (extra labelled information sent with a request), for example /api to one pool and /images to another. Envoy, NGINX, HAProxy and the AWS Application Load Balancer work at layer 7. Large systems stack them: L4 in front, spreading connections across many L7 proxies. Then come four design questions. Which algorithm picks the server: round robin, least requests, power of two choices or consistent hashing? How do health checks find a dead server, and how fast? How do you remove a server without cutting requests off halfway (connection draining)? And how do you keep the load balancer itself up? The answer to the last one is several machines that share one address.
readDesign a Unique ID Generator in Distributed Systems: Snowflake, UUIDv7, Ticket Servers and Segments
A unique ID generator gives every new record a number that is never used twice, across many servers, with no central bottleneck. First, some terms. A bit is one binary digit, 0 or 1; 64 bits hold any whole number up to about 9.2 billion billion. A BIGINT is the database column type that holds a 64-bit whole number. An epoch is the fixed starting moment that a time count is measured from. The main options are these. A database counter (auto-increment: the database adds 1 for each new row) is simple. But it runs on one machine, so if that machine fails, nobody gets a new ID. Several databases can share the work by stepping: one gives odd numbers, one gives even numbers, which is how Flickr's ticket servers work. A UUID is a 128-bit ID. Version 4 is random, so any machine can make one without asking other machines, but is twice the size of a BIGINT and scatters new rows all over an index. Version 7, defined in RFC 9562 (an official internet standard), puts a 48-bit millisecond timestamp first, so new IDs land next to each other. Twitter's Snowflake builds a 64-bit number from three parts: 41 bits of milliseconds since a chosen epoch, a 10-bit machine ID, and a 12-bit sequence counter. That gives 4,096 IDs per millisecond per machine and about 69.7 years before the time bits run out. Instagram used the same idea inside PostgreSQL, with 13 bits for a shard number (a shard is one of the many databases that together hold the data) and 10 bits for the counter. Segment allocation hands each server a block of numbers from a database, such as 1,000 at a time, so the database is asked once per block, not once per ID. The hard parts are giving every machine a different machine ID, surviving a clock that jumps backwards, and handling more than 4,096 requests in one millisecond.
readDesign an AI Agent for Customer Support: Tools, Refund Approval, Human Handoff and pass^k Evaluation
An AI agent for customer support is an LLM (large language model) that runs in a loop. It reads a customer's message, looks things up, and takes actions such as a refund. Code and people check every risky step. Design it in seven parts. First, channels: a chat widget and email, both feeding one ticket system, where each customer conversation is one 'ticket'. Second, an intake router. This is a small, cheap model or a classifier (a program that sorts text into fixed groups). It labels each message with an intent, meaning what the customer wants (where is my order, refund, change address), and a risk level. It sends high-risk cases, such as fraud or legal threats, straight to a person. Third, a help-centre search with RAG (retrieval-augmented generation: find the right policy articles first, then have the model answer from them), so policy answers quote the shop's own rules with a citation. Fourth, account tools: order lookup, refund, address change. A tool call is the model asking code to run a named function with arguments. Each tool sits behind an API (a fixed doorway through which programs ask for data or actions) with least privilege, which means it can reach only what it needs, and with limits per tool. Fifth, guardrails, which are hard rules written in code: identity verified before any account action, refunds only inside policy and under a limit, and a step and cost ceiling on every run. Sixth, people: refunds above the limit pause and wait for a human to approve, and any ticket the agent cannot finish goes to a human agent with the full history attached. Seventh, state and proof: each run is saved step by step so it can resume after a crash, every money action carries an idempotency key (a fixed label for one action, so a repeat is spotted and skipped), so a retry never pays twice. Every step is recorded as a trace (a full log of the run). Changes are tested offline, before going live, on real past tickets with pass^k, the chance the agent gets the same task right on all k tries.
readBanking System Design Interview: Double-Entry Ledger, Holds, Transfers and Sharding
A banking system keeps every account in a double-entry ledger. A ledger is the bank's book of every money movement. Double-entry means each movement is written as at least two entries: a debit on one account and a credit on another. For a customer's account, a debit takes money out and a credit puts money in. The entries of each movement add up to zero, so money never appears or vanishes. It only moves. The ledger is the source of truth: a balance is the sum of an account's entries. For speed, a balance row sits next to the entries. It is updated in the same database transaction, a group of changes the database applies all together or not at all. Each account shows two balances. The ledger balance counts only posted, meaning final, entries. The available balance also subtracts holds. A hold reserves money for a payment that was approved but not yet collected, such as a card payment. A transfer inside the bank is one transaction. First, lock both balance rows, lowest account id first, so two transfers never wait for each other forever. Second, check the available balance. Third, insert both entries. Fourth, update both balances. Fifth, write an event to an outbox table, a table of messages for other services. A transfer to another bank first moves the money into a suspense account, an internal account for money in transit. Then it goes out through a clearing network, the system that carries payments between banks, such as NEFT, IMPS or RTGS. It completes or comes back later. Entries are never changed or deleted. A mistake is fixed with a reversing entry, a new entry that cancels the old one. The write path is strongly consistent: every balance check that decides whether money can move reads the latest committed (made final) balance on the primary database, the one server that takes writes. Accounts are sharded, meaning split across database servers, by account id. Every commit is copied to a standby server before the customer is told it worked, so a crash loses no finished transfer.
readDesign a CDN (Content Delivery Network): Routing, Caching, Purge and Origin Shield
A CDN (content delivery network) is a set of servers in many cities. Each server keeps copies of a website's files so it can answer users without asking the website's own server. Keeping copies like this is called caching. The website's own server is called the origin. A group of CDN servers in one city is a PoP (point of presence). A user reaches a nearby PoP in one of two ways. With DNS mapping, the CDN's DNS server (the internet's phone book, which turns a name into an address) picks a PoP for the user. With anycast, every PoP announces the same IP address, and internet routing carries each user to one of them. At the PoP, the server ends TLS, the encryption that makes HTTP into HTTPS. Then it builds a cache key: the label it files a stored copy under, usually the host name, the path and some query parameters (the part of a URL after the ?). If it has a copy, that is a hit, and it answers at once. If not, that is a miss. A miss goes to a regional cache layer called the origin shield, and only the shield asks the origin. The shield also does request collapsing: many misses for the same file become one request to the origin. The origin's Cache-Control header (a line of extra information sent with the response) says how long a copy stays fresh. stale-while-revalidate lets the edge send an old copy at once while it fetches a new one. stale-if-error lets it send an old copy when the origin is down. When content changes, a purge removes the old copies from every PoP. File names that change with the content (app.3f9a.js) avoid most purges. The numbers to watch are the hit ratio (the share of requests answered from a copy), the load left on the origin, and how fast a purge reaches every PoP.
readMLOps System Design Interview: Design an ML Platform for Training, a Model Registry, Safe Rollout and Monitoring
MLOps means running machine learning models in real products with the same care as any other software. An MLOps platform is the shared system a company uses to train, store, release and watch many models. Design it as one loop with five parts. First, data: every training set is a fixed, named copy, so a model can be rebuilt later. Second, a training pipeline: a chain of steps (check the data, build the inputs, train, test) that runs on a schedule or when an alarm fires. Each step runs inside a container image, a sealed package that holds the code, Python and every library. The library versions are pinned, which means fixed. Third, a model registry: a catalogue where every trained model becomes a version that never changes. It also holds aliases, which are movable names such as champion that point at one version. Serving means answering live requests with a model, and serving loads the alias, so a rollback just points the alias back. Fourth, rollout. A new version must beat the current one on held-out rows: examples kept aside and never used in training, so they give a fair test. The rule for winning is written down in advance. Then the new version runs in shadow: it answers live requests, but nobody uses its answers. Then it runs as a canary: it really answers a small share of requests. Fifth, monitoring. Input drift means the live inputs no longer look like the training inputs. Drift alarms are cheap and fast, but they do not prove harm. So the platform also checks real accuracy on a small labelled sample: rows where the true answer is now known. A feature is one input value for a model, such as a customer's purchases in the last 30 days. A feature store gives training and serving one definition of each feature. Its point-in-time join gives each training row only the values that existed at that row's own time, so training never sees the future. Around it all sit multi-tenancy (many teams sharing one platform safely) and quotas (a cap on how much each team may use, especially of GPUs, the costly chips that speed up training).
readDesign a Distributed Cache: Sharding, Eviction, Hot Keys and Stampedes
A distributed cache is a set of machines that each hold part of the cached data in memory (RAM), in front of a database that stays the source of truth. Each item is a key and a value. The app usually follows cache-aside: read the cache first; on a miss, read the database and put the value in the cache; on a write, update the database and delete the cached key. Keys are spread over the cache machines with consistent hashing. A hash function turns a key into a number; each machine owns many points on a ring of hash values (virtual nodes), and a key belongs to the first point clockwise from its own hash. Adding a machine then moves only about 1/N of the keys. Each machine evicts (removes) old items when it is full, usually by LRU, least recently used, which a hash map plus a doubly linked list does in constant time. Every item also gets a TTL, a time to live, after which it expires. The hard parts are failures and load spikes. A dead machine turns all its keys into misses that hit the database at once. One very popular key can overload one machine. When a popular key expires, thousands of requests can miss together and flood the database; this is a cache stampede. Facebook's memcache paper fixes stampedes with leases: only one client per key is allowed to refill it, and the others wait briefly and retry. In their measurement, peak database reads for such keys fell from 17,000 to 1,300 a second.
readDesign a Hotel Reservation System: Inventory, Double Booking, Holds and Overbooking
A hotel reservation system keeps one inventory row per hotel, per room type, per night. Each row holds two counts: how many rooms of that type exist, and how many are already reserved. Booking a stay adds 1 to the reserved count of every night in the stay, inside one database transaction. A transaction is a group of changes that the database applies all together or not at all. To stop double booking, the check and the write must happen as one step. The simplest way is one guarded UPDATE: add 1 only if reserved + 1 is still within the limit, written as a WHERE condition in the SQL statement. If any night is full, the update changes fewer rows than the stay has nights, and the whole booking is rolled back (undone). A row lock (SELECT ... FOR UPDATE, which makes other bookings for those rows wait) or a version number check also works. The booking API is idempotent: the app sends a unique key with each booking, so a retry after a network error returns the first answer instead of holding a second room. A new booking starts as HELD with a timer of a few minutes. It becomes CONFIRMED when the card is authorized, or EXPIRED, which gives the room back. Hotels may allow overbooking, a set percentage above the real room count, based on how many guests usually do not arrive. Browsing reads a cache, a fast copy kept in memory that may be a few seconds old; only the inventory database decides a booking. Data is sharded by hotel_id, meaning it is split across database servers by hotel, so every row one booking touches lives on one server.
readDesign a News Aggregator like Google News: Feeds, Dedup, Story Clustering and Ranking
A news aggregator such as Google News works as a pipeline. First, fetchers collect new articles from publishers. A publisher's feed is a small file (RSS or Atom) that lists its newest articles. The fetchers poll each feed on a schedule. To poll is to ask again and again whether anything changed. Where a publisher supports WebSub, a hub pushes new items to us instead. The fetchers obey robots.txt, a file in which each site lists what crawlers may fetch, and limit how often they hit each site. Second, each article is cleaned and checked for duplicates in four steps, from cheapest to most costly: same id or URL, same text hash (a short code computed from the text, equal only for equal texts), nearly identical text (SimHash, a 64-bit fingerprint where copies differ in at most 3 bits), and same event. The last step, story clustering, keeps every article but groups those about one event into a single story. Third, a ranking job scores each story about once a minute. A good score mixes freshness (score halves every few hours), importance (how many publishers cover it) and location and language. It writes one ranked list per edition and section to a cache. Fourth, each page view reads those shared lists and re-ranks a few hundred candidates for the reader, using their interests. Signed-out readers get a cached global page. Copying every story into each reader's own feed would cost trillions of writes a day, so the design ranks shared lists once and personalizes on read.
readDesign a Top-K System (Heavy Hitters): Count-Min Sketch, Heaps and Windows
A top-k system finds the k items that occur most often in a stream of events, such as the 10 most-viewed videos in the last hour. The items that occur most often are called heavy hitters. The usual design has two paths. The fast path reads view events from a log such as Kafka, sends each video to one counting worker by hashing its id, and counts views per time window. To save memory, each worker counts with a count-min sketch: a small table of counters, d rows by w columns, where each row hashes the video to one counter. To add a view, add 1 to one counter in every row. To read a count, take the smallest of those d counters. The answer is never too low. With width w = e/ε and depth d = ln(1/δ), it is too high by more than ε times the total number of views only with probability δ. A sketch cannot list its videos, so each worker also keeps a min-heap of size k: a small list that always knows its smallest member, so a new video replaces that member when its estimate is larger. The workers send their lists to one merger, which builds the global top k. The slow path stores every view and recounts them exactly every hour, then replaces the estimates. The traps are merging lists from partitions that split a video's views (the true winner can vanish), sliding windows, and one viral video overloading one worker.
read