Discord System Design Interview: Real-Time Chat and Voice at Guild Scale
Discord has published that its real-time infrastructure runs on roughly 400 to 500 Elixir machines pushing on the order of 26 million WebSocket events to clients every second, with about 12 million concurrent users at the time of that writing. Its voice platform has served more than 2.6 million concurrent voice users across 850-plus voice servers, pushing over 220 Gbps of egress. The message store holds trillions of rows and was migrated from Apache Cassandra to ScyllaDB, dropping from 177 nodes to 72.
Discord is two hard systems wearing one app. The first is a real-time gateway: every client holds a WebSocket, and events like new messages, typing, and presence must fan out to everyone watching a channel within a guild, sometimes millions of members. Discord models each guild as a single Elixir process on the BEAM that routes events to per-connection session processes, which works beautifully for small servers and becomes a fanout bottleneck for huge ones. The second system is durable message storage. Messages are immutable, append-heavy, and read by channel in reverse-chronological order, so Discord partitions them by channel and a time bucket and stores them in a wide-column store, first Cassandra and now ScyllaDB, fronted by a Rust data service that coalesces duplicate reads. On top of that sit voice servers running a homegrown selective forwarding unit over UDP and WebRTC, plus per-user read state, mentions, and unread counts. A strong answer treats fanout, storage partitioning, and voice as three separate scaling problems with three different solutions.
Asked at: Asked at Discord, and used at Meta, Google, Slack, and other companies that run real-time messaging or presence systems. It also shows up in senior interviews under the generic 'design a chat app' or 'design a real-time messaging system' prompt.
Why this question is asked
It forces a candidate to separate the read path, the write path, and the push path instead of blurring them into one 'chat service'. The interesting tension is fanout: a message write is cheap, but delivering it to every online member of a large guild is expensive, and the naive one-process-per-guild design falls over exactly where the product is most visible. It also rewards candidates who know that immutable, time-ordered data wants a wide-column store with careful partition keys rather than a relational schema, and who can reason about hot partitions, tombstones, and per-user state that cannot be denormalized into the shared message rows.
Requirements
Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.
Functional requirements
- Users send and receive messages in text channels that belong to guilds (servers), with real-time delivery to all online members who can see the channel
- Load recent message history for a channel and scroll back through older messages in reverse-chronological order
- Support guilds ranging from a handful of friends to communities with millions of members
- Show presence (online, idle, offline) and typing indicators to members of a guild
- Track per-user read state: which channels have unread messages, unread counts, and mentions directed at the user
- Join a voice channel and exchange low-latency audio and video with other participants
- Support edits, deletions, reactions, replies, and attachments on messages
- Deliver messages and mentions reliably even when a user is briefly offline, so nothing is silently lost
Non-functional requirements
- Real-time delivery latency in the tens of milliseconds for message and presence fanout
- Very high WebSocket connection count, on the order of millions of concurrent persistent connections
- Message store must scale to trillions of rows with predictable low read latency by channel
- Voice must be low-latency and loss-tolerant, favoring UDP with graceful handling of dropped packets
- High availability: a single node or voice server failure must not take down guilds or ongoing calls
- Elastic fanout so that a message to a giant guild does not stall delivery for small guilds sharing the same infrastructure
- Durability for messages once acknowledged, and correctness of per-user read state under concurrent activity
Back-of-envelope scale estimates
Show your math. Pulling numbers from thin air signals you have not thought about the load.
Concurrent gateway connections
~12 million (published)
Discord's Elixir engineering writeup cited about 12 million concurrent users served by 400 to 500 Elixir machines. Each connection is a long-lived WebSocket with its own session process.
WebSocket events pushed to clients
~26 million per second (published)
Reported in Discord's Elixir at scale material. This is the outbound fanout rate, not the inbound message rate, which is far smaller because one message can fan out to thousands of sessions.
Inbound messages vs outbound deliveries
1 message can become 1M deliveries (Derived)
Discord's own tracing post notes a message to a guild with a million online members spawns roughly a million forwarding operations, one per session. Fanout amplification, not ingest, is the dominant cost.
Total stored messages
Trillions of rows (published)
Discord states the message cluster holds trillions of messages. Cassandra crossed a billion messages in 2017 and grew from there; treat the exact current count as an estimate.
Message store latency after ScyllaDB
p99 read ~15ms, insert ~5ms (published)
Discord reported p99 message reads improving from 40 to 125ms on Cassandra to about 15ms on ScyllaDB, and inserts from 5 to 70ms down to about 5ms.
Concurrent voice users
2.6 million+, 220 Gbps egress (published)
Discord's WebRTC post reported more than 2.6 million concurrent voice users, over 220 Gbps egress and 120 Mpps, served by 850-plus voice servers across 13 regions.
High-level architecture
A client opens a WebSocket to the gateway and authenticates. The gateway spins up a per-connection session process on the BEAM (the Erlang virtual machine that Elixir runs on). That session subscribes to the guilds the user is a member of. Each guild is itself a process living on one of a set of guild nodes; the guild process is the routing hub that knows which sessions are currently watching which channels. When a user sends a message, it goes over that user's WebSocket to the API layer, which persists the message to the storage tier and then hands it to the owning guild process. The guild process fans the event out to the session processes of every online member who can see the channel, and each session writes it down its own socket. For very large guilds this single guild process becomes a hotspot, so Discord introduced Manifold to spread the sending work across nodes and cores, and relays to hold sessions on behalf of enormous guilds. Message persistence lands in a wide-column store (ScyllaDB today, Cassandra before it) partitioned by channel and time bucket, fronted by a Rust data service that coalesces concurrent reads of the same channel so a viral message does not stampede the database. Voice runs on a separate path entirely: the gateway hands the client a voice server in a nearby region, and the client speaks UDP and WebRTC directly to that server's selective forwarding unit, which relays media to the other participants without routing it through the chat backend.
In a real interview, sketch this on the whiteboard before diving into any single box.
Core components
Walk through each service. The interviewer wants to hear what each one owns, not just the names.
Gateway (WebSocket layer)
Terminates millions of long-lived WebSocket connections, built on Elixir and Cowboy. Each connection gets a session process that authenticates the client, tracks which guilds and channels it cares about, and writes events down the socket. Because it is a BEAM process, a crash on one connection is isolated and cheap to restart.
Guild process
One Elixir GenServer per guild acting as the central router for that server. It holds the set of active sessions and dispatches message, presence, and typing events to them. This model is elegant for small and medium guilds and becomes the primary scaling challenge for guilds with hundreds of thousands or millions of members.
Relays for large guilds
For very large guilds, relay processes maintain session connections on the guild's behalf and take over fanout and permission checks, with each relay handling a bounded slice of the membership. This keeps a single guild process from having to talk directly to millions of sessions.
Message data service (Rust)
An intermediary between the API and the database, written in Rust. It routes requests by channel id using consistent hashing and performs request coalescing, so many simultaneous reads of the same channel collapse into a single database query. This is what protects the store when an @everyone announcement makes a whole community open the same channel at once.
Message store (ScyllaDB)
A wide-column store holding trillions of messages, partitioned by channel and a static time bucket, with Snowflake ids as clustering keys so rows sort chronologically. Discord migrated here from Cassandra to escape JVM garbage collection pauses and reduce node count, cutting p99 read latency to around 15ms.
Voice servers and SFU
Standalone servers, separate from the chat backend, each with a signaling component and a media relay called a selective forwarding unit written in C++. Clients send UDP and WebRTC media to the assigned server, and the SFU forwards streams to the other channel participants. Region selection places users on a nearby server to keep latency low.
Read state and presence service
Tracks per-user, per-channel state that cannot be baked into shared message rows: last-read message, unread counts, and mentions. Presence tracks online, idle, and offline status and is published through the guild process to interested sessions.
Data model
Pick the right store per table. Justify each choice with the access pattern, not by reflex.
messagespartition_key (channel_id, bucket)message_id (Snowflake, clustering)author_idcontentflagsreferenced_message_idPartitioned by channel plus a static time-window bucket so a single busy channel does not create an unbounded partition. Snowflake message ids are time-sortable, so a channel read is a bounded slice within one or a few partitions, read in reverse order.
channelschannel_idguild_idtypenamepermission_overwritespositionA channel belongs to a guild and carries permission overwrites that decide who can read it. Fanout uses these permissions to skip sessions that cannot see the channel.
guild_membersguild_iduser_idrolesnickjoined_atMembership and roles for a guild. For huge guilds this is far too large to load into a client at once, so member lists are streamed and lazily materialized rather than fully denormalized.
read_stateuser_idchannel_idlast_read_message_idmention_countupdated_atPer-user state, keyed by user and channel, kept separate from the shared message rows. Unread is computed by comparing last_read_message_id against the newest message id in the channel.
voice_statesguild_idchannel_iduser_idsession_idvoice_server_idmute_deaf_flagsTracks who is in which voice channel and which voice server holds their media session. This is transient state, rebuilt on reconnect, not long-term durable data like messages.
Deep dives
These are the conversations the interviewer is steering you toward. Practice each one until you can talk through it without notes.
The guild process and the fanout bottleneck
Discord models each guild as one GenServer on the BEAM. When something happens, a message, a presence change, a typing event, that guild process pushes it to every relevant session process. For a friend server with 50 people this is trivial. For a guild with tens of thousands of concurrent members, Discord found that publishing a single event could take anywhere from 900ms to 2.1 seconds, because the process was serially handing off to thousands of recipients across many nodes. The single process is both the strength of the model (a clean, isolated routing point per server) and its scaling wall. The fixes were to parallelize the sending work and, for the largest guilds, to stop having one process talk to every session directly.
Manifold and cross-node message distribution
Manifold is Discord's open-source library for fanning a message out to many process ids that live on many nodes. Instead of the guild process calling send once per recipient, Manifold groups the recipient pids by their remote node, sends one batch to a partitioner on each of those nodes, and the partitioner then consistently hashes the pids across a pool of workers that do the final local delivery. The effect is that the originating process makes at most one send per remote node rather than one send per recipient, which moves the fanout work off the hot guild process and onto the machines where the recipients actually live. It preserves the ordering guarantees of the underlying send while spreading load across cores.
Node lookup, fastglobal, and the registry stampede
Sessions constantly need to find the guild process for a given guild, which means a lookup of guild id to node. Doing this through a normal process or ETS table became a hot path under millions of sessions. Discord used a technique they packaged as fastglobal, which exploits the fact that when a function always returns the same constant data, the BEAM stores it in a read-only shared heap so any process can read it without copying, turning a microsecond lookup into a nanosecond one. Solving the lookup speed then exposed a second problem: with the back-pressure of the slow lookup gone, roughly five million sessions could stampede the ten registry processes at once. Discord added a semaphore built on atomic ETS counters to bound concurrency into those processes and prevent cascading overload.
Partitioning trillions of messages and the hot partition problem
Messages are immutable and almost always read by channel, newest first, so the natural partition key is the channel. But a single popular channel would produce an unbounded partition, and wide-column stores hate unbounded partitions. Discord partitions by channel plus a static time bucket, so each partition covers one channel for one time window, and a channel read touches one or a few bounded partitions. Snowflake ids serve as the clustering key so rows sort chronologically inside a partition. The remaining pain point is the hot partition: a huge public channel gets orders of magnitude more traffic than a small friend server, and a big announcement can trigger a stampede of concurrent reads on the same partition. That is exactly what the request-coalescing data service in front of the database is for.
Cassandra pain, tombstones, and the move to ScyllaDB
Discord ran messages on Cassandra from 2017 and hit two recurring problems as it grew to 177 nodes and trillions of rows. First, JVM garbage collection pauses made latencies unpredictable and required constant tuning, and the compaction and repair cycle was operationally heavy. Second, deletes create tombstones, and reading across many tombstones is expensive, which interacts badly with hot partitions. Discord moved to ScyllaDB, a C++ reimplementation of the Cassandra data model with no garbage collector and a shard-per-core architecture. Combined with the Rust data services that coalesce reads, this cut the cluster from 177 nodes to 72, raised per-node storage, and dropped p99 reads from the 40 to 125ms range down to around 15ms. The migration itself was done with a Rust migrator that firehosed data at about 3.2 million rows per second and finished in nine days instead of the estimated three months.
Voice: separate servers, SFU, UDP, and region selection
Voice does not go through the chat backend at all. When a user joins a voice channel, the gateway assigns a voice server in a nearby region and hands the client its address and encryption keys. The client then speaks UDP and WebRTC directly to that server. Each voice server runs a signaling component and a selective forwarding unit written in C++. The SFU does not mix or transcode audio; it simply forwards each participant's encrypted stream to the others, which keeps CPU cost low and latency down. UDP is chosen deliberately because for real-time audio it is better to drop a late packet than to wait for a retransmit. Region selection and having 850-plus servers across many data centers is what keeps the round trip short regardless of where participants are.
Per-user read state, mentions, and unread at scale
The shared message rows are the same for everyone, but unread state is personal, so it cannot be denormalized into those rows. Discord keeps read state keyed by user and channel: the id of the last message the user has read, an unread mention count, and a timestamp. Whether a channel is unread is a cheap comparison of the user's last-read message id against the channel's newest message id, both of which are time-sortable Snowflakes. Mentions are computed at send time by parsing the message for user, role, and everyone references and incrementing the mention counters for the affected users. The tricky part at scale is that a mention of a role in a huge guild can touch a large number of users' read state, so this write amplification is bounded and handled carefully rather than done inline on the send path for every recipient.
Trade-offs to discuss
Every senior interviewer expects you to surface at least 3 of these. Pick the decisions, state the alternatives, and justify your choice.
One process per guild versus one process per channel or per shard
A single guild process is a clean routing hub and keeps all of a server's state in one place, which is simple to reason about and cheap for the vast majority of guilds. The cost is that a few enormous guilds turn that one process into a bottleneck, which Discord patches with Manifold and relays rather than abandoning the model.
Wide-column store (ScyllaDB) versus a relational database for messages
Messages are immutable, append-heavy, and read by channel in time order, which maps directly to a partitioned wide-column model with time-sortable clustering keys. A relational store would struggle to scale writes and partitions to trillions of rows, but the tradeoff is losing joins and transactions, so per-user state and relationships live elsewhere.
ScyllaDB versus staying on Cassandra
ScyllaDB keeps the Cassandra data model and query semantics but drops the JVM, eliminating garbage collection pauses and cutting node count and tail latency. The tradeoff was a large, risky migration of a live trillions-of-rows cluster, which Discord accepted because the operational burden of Cassandra was growing faster than the team.
UDP with dropped packets versus reliable TCP for voice
Real-time audio values freshness over completeness, so dropping a late packet and moving on sounds better than stalling for a retransmit. TCP's in-order reliable delivery would add head-of-line blocking and latency, which is the wrong tradeoff for a live call.
Selective forwarding unit versus a mixing MCU for voice
An SFU just relays each stream and lets clients mix, which keeps server CPU low and scales cheaply across many participants. A mixing unit would cut client bandwidth but burn far more server CPU per channel, so Discord chose forwarding for cost and performance.
Request coalescing in a data service versus letting clients hit the database
Putting a Rust data service in front of the store lets Discord collapse a stampede of identical reads into one query, protecting hot partitions during viral moments. The cost is an extra hop and a stateful routing layer, which is worth it because the alternative is the database melting under a popular announcement.
How Discord actually does it
Discord has documented this architecture in unusual detail. The Elixir gateway, guild processes, and session model are described in their Elixir at scale writeups, and the fanout libraries Manifold and fastglobal are open source on Discord's GitHub. The message storage journey from MongoDB to Cassandra to ScyllaDB, the channel-plus-time-bucket partitioning, the tombstone and hot partition pain, the Rust data services with request coalescing, and the nine-day migration at roughly 3.2 million rows per second are all covered in How Discord Stores Trillions of Messages. Voice, including the C++ selective forwarding unit, UDP and WebRTC transport, region selection, and the 2.6 million concurrent voice user figure, is covered in their WebRTC post. Their 2026 tracing post is where the million-child-spans-per-message framing of fanout comes from.
Sources
- How Discord Stores Trillions of Messages (Discord Engineering)
- How Discord Scaled Elixir to 5,000,000 Concurrent Users (Discord Engineering)
- Real-time communication at scale with Elixir at Discord (elixir-lang.org)
- How Discord Handles 2.5 Million Concurrent Voice Users Using WebRTC (Discord Engineering)
- Tracing Discord's Elixir Systems Without Melting Everything (Discord Engineering)
Lessons to study before this interview
If any of these topics are fuzzy, the interviewer will catch it. Each lesson is 15 to 60 minutes with diagrams, code, and a quiz.
Related system design interview questions
Practice these next. They lean on the same core building blocks as Discord.