DynamoDB System Design Interview: Designing a Distributed Key-Value Store That Serves Millions of Requests per Second at Single-Digit-Millisecond Latency

A client issues an operation against a table, addressing an item by its primary key, and speaks to the service through a stateless request layer. The first job of that layer is routing: it takes the partition key, hashes it, and looks up which partition owns that hash range in the partition metadata. That lookup tells it which storage nodes hold the replicas for the item. The request router is not the source of truth for data; it is a thin, horizontally scalable tier whose only job is to authenticate the caller, check that the request stays within its allocated throughput (admission control), find the right partition, and forward the operation. Because routing is a pure function of the key plus a metadata map, you can run as many request routers as you need and any of them can serve any request. Data is partitioned by consistent hashing. The keyspace is a ring, each key hashes to a point on the ring, and each partition owns a contiguous arc of that ring. When a partition gets too big or too hot, it splits, and the metadata map is updated so future requests route to the new owners. Each partition is replicated, typically three ways, across separate availability zones so that the failure of a disk, a machine, or an entire zone never takes the data offline. Production DynamoDB designates one replica per partition as the leader. Writes go to the leader, which appends to a write-ahead log and replicates to the other members; a write is acknowledged once a quorum has durably accepted it. Strongly consistent reads are served by the leader, which knows it has the latest committed value, while eventually consistent reads can be served by any replica and may lag slightly. On each storage node, an item lives in a local storage engine, commonly a log-structured merge tree (LSM) or a B-tree, that turns random-looking key access into efficient sequential disk writes and indexed reads. Writes first hit a durable log and an in-memory structure, then get flushed and compacted in the background. This is what lets the node absorb a high write rate while still answering point reads quickly. Secondary indexes are maintained as their own partitioned structures that are updated asynchronously from the base table change stream, which is why a global secondary index is eventually consistent with the item you just wrote. Underneath all of this is a control plane that the data path does not sit on directly. It tracks node membership and health, decides when a partition should split or move, drives the actual data movement while the partition stays online, and updates the metadata map that the request routers consult. Membership and failure information propagate through a gossip-style mechanism in the original Dynamo design and through a more centralized, quorum-backed metadata service in production DynamoDB, but the principle is the same: the fast path stays simple and local, and the slow, global coordination is pushed off to a separate system that is not in the critical path of a get or a put.

In a real interview, sketch this on the whiteboard before diving into any single box.