Spotify System Design Interview: Streaming Audio, Discover Weekly, and Billions of Play Events
Spotify serves a catalog of roughly 100 million tracks to hundreds of millions of monthly users. Each track is encoded at several bitrates, so the object store holds hundreds of millions of audio files. The play event system has publicly handled peaks above 8 million events per second and more than 350 TB of raw event data per day, and that firehose is what pays artists and drives Discover Weekly.
Spotify looks like a simple play button, but the interesting engineering sits underneath it. Audio is not generated on the fly the way video often is. Every track is pre-encoded into a handful of Ogg Vorbis bitrates, chunked, and pushed to a CDN so the client can start playback in a few hundred milliseconds and switch quality as the network changes. The catalog and metadata for tracks, albums, artists, and playlists need fast lookups and search across a huge corpus. Personalization is the other half of the product: collaborative filtering and content signals feed offline batch jobs that precompute recommendations like Discover Weekly, which are then served from low latency stores. Tying it all together is an event pipeline that captures every play, skip, and save. That pipeline feeds royalty accounting, analytics, and the recommendation models, so it has to be durable and ordered enough to trust with payments. The hard parts to talk about in an interview are audio delivery and caching, the recommendation pipeline that splits offline compute from online serving, and the event delivery backbone that Spotify famously moved from Kafka to Google Cloud Pub/Sub.
Asked at: Asked at Spotify, Apple, Amazon, and Netflix, and common in senior interviews at any company that streams media or runs a large recommendation surface. It also shows up at music and podcast startups that want to see if you understand media delivery.
Why this question is asked
This problem rewards a candidate who can separate a media delivery problem from a data platform problem and handle both. It tests whether you understand that streaming audio is mostly a caching and CDN story rather than a compute story, whether you can design a recommendation system that splits heavy offline batch training from cheap online serving, and whether you can build an event pipeline that is trustworthy enough to drive royalty payments. There is a lot of surface area, so it also tests prioritization. A strong answer picks two or three subsystems and goes deep instead of drawing every box.
Requirements
Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.
Functional requirements
- Stream any track from a large catalog with fast startup and no audible gaps
- Adapt playback quality to the user's network and device without stalling
- Let users search tracks, albums, artists, and playlists and browse the catalog
- Create, edit, and share playlists, including collaborative playlists edited by multiple users
- Generate personalized recommendations such as Discover Weekly, Release Radar, and the home feed
- Record every play, skip, save, and seek for royalties, analytics, and model training
- Support offline downloads of tracks and playlists within licensing and DRM constraints
- Support social features like following artists and friends and seeing what they play
- Enforce free versus premium entitlements, including bitrate caps and offline access
Non-functional requirements
- Playback start latency in the low hundreds of milliseconds for popular tracks
- High availability for playback even when personalization or search is degraded
- Event durability strong enough to trust for royalty accounting
- Global reach with content served close to the listener
- Recommendations that refresh on a predictable cadence, for example weekly
- Elastic scale for event volume that spikes during launches and big releases
- Privacy and compliance controls for listening history and personal data
Back-of-envelope scale estimates
Show your math. Pulling numbers from thin air signals you have not thought about the load.
Catalog size
~100M tracks
Spotify has publicly cited a catalog around 100 million tracks. Older engineering posts referenced 30M songs, so this grew over a decade. Treat the current number as an estimate.
Audio files in storage
~300M objects
Derived. 100M tracks times roughly 3 stored bitrates (for example 96, 160, 320 kbps Ogg Vorbis) is about 300M audio blobs, before counting per chunk splitting and podcast content.
Raw audio storage
~1 to 2 PB
Derived estimate. A 4 minute song is roughly 4 MB at 128 kbps and 12 MB at 320 kbps. Averaging a few MB per encoding across 300M objects lands in the low petabytes for the primary catalog.
Peak event throughput
>8M events/sec
Spotify's own 2019 writeup reported peaks above 8 million events per second through the delivery system. Earlier posts cited around 1.5M events/sec, so growth is steep. Published figure, treat current peak as higher.
Daily event data
~350+ TB/day
The 2019 event delivery post reported more than 350 TB of raw events per day flowing into storage and BigQuery. Published as of that writeup.
Discover Weekly reach
billions of streams/week
Discover Weekly is a per user weekly playlist. Across hundreds of millions of users it drives billions of streams weekly. Order of magnitude estimate from public reporting.
High-level architecture
A client opens a track and first talks to the backend to resolve entitlements and get a signed URL or manifest for the audio. The audio itself does not come from application servers. It sits pre-encoded in object storage and is fronted by a CDN, so the client fetches encrypted audio chunks from the nearest edge, starts playback quickly, and prefetches the next chunks while adapting bitrate to the measured network. Metadata for the track, album, artist, and any playlist comes from catalog services backed by fast key value and search stores, not from the audio path. While the user listens, the client and backend emit events for every play, skip, seek, and save. Those events flow into an event delivery system where each event type gets its own topic on Google Cloud Pub/Sub, then gets deduplicated, optionally encrypted for privacy, and landed in Cloud Storage and BigQuery in immutable hourly buckets. Batch jobs read that history to train collaborative filtering models and precompute personalized playlists like Discover Weekly, writing results into low latency serving stores. When the user opens the home screen or the Discover Weekly playlist, a serving layer reads those precomputed recommendations and blends them with fresh signals. Playback stays up even if search or personalization is degraded, because the audio path and the data platform are deliberately decoupled.
In a real interview, sketch this on the whiteboard before diving into any single box.
Core components
Walk through each service. The interviewer wants to hear what each one owns, not just the names.
Audio encoding and packaging pipeline
An offline pipeline ingests a master track and encodes it into several Ogg Vorbis bitrates, commonly around 96, 160, and 320 kbps, plus AAC where a device needs it. It splits each encoding into chunks and encrypts them, then writes the results to object storage so nothing is transcoded at request time.
CDN and audio storage
Encrypted audio chunks live in object storage and are served through a CDN that caches popular content on edge servers near listeners. This keeps startup latency low and keeps the long tail of the catalog reachable from origin. The audio path is intentionally separate from metadata and personalization services.
Catalog and metadata services
These services own tracks, albums, artists, and playlists and answer lookups for the player and browse surfaces. They are read heavy, so they lean on caching and denormalized views. Search runs on an inverted index over the catalog for typeahead and full queries.
Playlist service
Handles user created, editorial, and algorithmic playlists. Collaborative playlists allow concurrent edits from multiple users, so the service needs an ordering and merge strategy that keeps everyone's view consistent. Algorithmic playlists like Discover Weekly are produced offline and stored per user.
Event delivery system
Captures every user and playback event and moves it reliably to storage and analytics. Each event type has its own Pub/Sub topic so a noisy high volume event cannot starve a business critical one such as the end of a song used for royalties. Data lands in immutable hourly buckets.
Recommendation and personalization pipeline
Offline jobs on the data platform train collaborative filtering and content models over listening history and write precomputed recommendations into serving stores. An online serving layer reads those results and blends in recent activity for the home feed and playlists. Heavy compute stays offline, serving stays cheap.
User profile and serving store
A low latency store, historically Cassandra and later Bigtable on Google Cloud, holds per user attributes and precomputed recommendations. Writes are cheap and frequent, reads are fast, and TTLs keep short lived signals from piling up. This is what the home and Discover surfaces read at request time.
Data model
Pick the right store per table. Justify each choice with the access pattern, not by reflex.
trackstrack_idtitleartist_idalbum_idduration_msaudio_manifest_refCore catalog entity. audio_manifest_ref points at the encoded chunk set in object storage rather than storing audio inline. Read heavy and cache friendly.
audio_filestrack_idbitratecodecchunk_indexstorage_keyencryption_key_idOne row per encoding chunk. A single track fans out to many rows across bitrates and chunks. storage_key resolves to the CDN and origin object.
playlistsplaylist_idowner_idtypeis_collaborativeupdated_attype distinguishes user, editorial, and algorithmic. Track membership lives in a child table ordered by position. Collaborative playlists need conflict handling on concurrent edits.
user_profileuser_idattribute_nameattribute_valuecontextttlWide row per user of derived listening attributes with short TTLs. Modeled for Cassandra style writes: cheap frequent upserts, fast point reads at serving time.
recommendationsuser_idsurfacegenerated_attrack_idsmodel_versionPrecomputed output such as Discover Weekly. surface identifies the playlist or feed. Written by offline batch jobs, read directly by the serving layer.
play_eventsevent_iduser_idtrack_idevent_typems_playedevent_timeAppend only. end of song events feed royalty accounting, so durability and dedup matter. Partitioned into immutable hourly buckets by event type in storage.
Deep dives
These are the conversations the interviewer is steering you toward. Practice each one until you can talk through it without notes.
Audio delivery and why streaming music differs from video
Music is small and repetitive compared to video, which changes the design. A three or four minute song at 320 kbps is only around ten megabytes, and the same popular tracks get played over and over, so the whole game is caching. Spotify pre-encodes each track into a few Ogg Vorbis bitrates offline, splits them into chunks, encrypts them, and stores them once. At play time the client resolves entitlements, gets references to the chunks, and pulls them from a CDN edge near the user. Because everything is precomputed there is no transcoding on the request path, which is what lets playback start in a few hundred milliseconds. Video adaptive streaming juggles far larger segments and steep bitrate ladders, while audio can hold a small working set of hot tracks resident at the edge and in the client cache. The client prefetches upcoming chunks and the next likely track so a skip feels instant.
Adaptive bitrate and on device caching
The client measures throughput and picks a bitrate that will not stall, dropping from 320 to 160 to 96 kbps when the network gets bad and climbing back when it recovers. Free and premium tiers also cap the ceiling. On device caching is a big lever because listening is repetitive, so the client keeps recently played and prefetched chunks locally and can serve a replay or a resume with no network at all. Offline downloads are the extreme version of this: the client fetches all chunks for a playlist ahead of time and stores them encrypted, then plays back within the license window. The tradeoff is storage on the device and key management, since the audio has to stay protected even when it is sitting on the phone.
The event delivery backbone and the move to Pub/Sub
Every play, skip, seek, and save becomes an event, and that stream is the lifeblood of royalties, analytics, and recommendations. Spotify originally ran this on Kafka and Hadoop on premises, then migrated the event delivery system to Google Cloud Pub/Sub as part of its cloud move. A defining choice is that each event type gets its own topic. That isolation means a noisy high volume event cannot delay a business critical one like the end of a song, which is what royalty accounting depends on. Events are deduplicated, sensitive fields are encrypted based on schema annotations, and data lands in immutable hourly buckets in Cloud Storage and BigQuery. The design favors liveness over lateness, so one stalled event type does not block the others. Different event classes get different delivery SLOs, from a few hours for critical events to longer windows for low priority ones.
The recommendation pipeline: offline batch, online serving
Recommendations split cleanly into a heavy offline half and a light online half. Offline, batch jobs read the full history of who listened to what and train collaborative filtering models, treating the huge user by track interaction matrix as the main signal. Co listening and co saving patterns place users and tracks in a shared latent space, and content signals from audio analysis help cover new or obscure tracks that lack interaction data. The output is precomputed per user, for example the thirty tracks in a Discover Weekly playlist, and written into a low latency serving store. Online, when the user opens the app, the serving layer just reads those precomputed lists and blends in recent activity. This split is what makes personalization affordable at hundreds of millions of users: the expensive matrix work runs on a weekly cadence in the data platform, and request time stays a cheap key value read.
Storing user profiles: Cassandra then Bigtable
Personalization needs a store that takes cheap frequent writes and answers fast point reads per user, which is exactly the Cassandra access pattern. Spotify built its User Profile Store on Cassandra, with a companion Entity Metadata Store for the slower changing attributes of artists, tracks, and playlists. Storm topologies consumed Kafka streams to compute short lived user attributes in near real time, while batch jobs on Hadoop bulk loaded derived data. As Spotify moved to Google Cloud it shifted much of this to managed Bigtable, which offers a similar wide column model without running clusters by hand, and built an autoscaler to tune node counts. The lesson for an interview is to justify the store from the access pattern: high write volume, per user reads, tunable consistency, and short TTLs on ephemeral signals rather than a relational schema.
Catalog, search, and collaborative playlists
Catalog reads dominate, so tracks, albums, and artists sit behind heavy caching and denormalized views, and search runs on an inverted index for typeahead and full text queries over a very large corpus. Playlists are their own problem. User and editorial playlists are ordered lists of track references, which is straightforward, but collaborative playlists let several people edit the same list at once. That needs a way to order concurrent edits so everyone converges on the same view, whether through a server assigned sequence, operational transforms, or conflict free merge semantics on the list. Algorithmic playlists like Discover Weekly are not edited at all, they are generated offline and stored per user, so they read like any other playlist even though a batch job produced them.
Royalties, correctness, and the value of ordered events
Royalty accounting is what raises the bar on the event pipeline from best effort analytics to something closer to a ledger. When a song is played past the threshold that counts as a stream, that event eventually turns into money owed to a rights holder, so the system cannot silently drop or double count it. That is why end of song events get isolation, deduplication, and a tighter delivery SLO than casual UI events. The pipeline does not need strict global ordering, but it needs durability and idempotent dedup so a retried or replayed event does not pay twice. Immutable hourly buckets help here because a bucket can be reprocessed deterministically. When you discuss this in an interview, separate the analytics path, where approximate and late is fine, from the money path, where you argue for at least once delivery plus dedup keys to reach effectively exactly once accounting.
Trade-offs to discuss
Every senior interviewer expects you to surface at least 3 of these. Pick the decisions, state the alternatives, and justify your choice.
Pre-encode fixed bitrates versus transcode on demand
Music is small and hot tracks repeat, so pre-encoding a few bitrates once and caching them beats paying transcode cost on every request. It uses more storage but gives fast startup and cheap serving. On demand transcoding only makes sense when the format space is huge and content is cold.
Offline precomputed recommendations versus fully online scoring
Training collaborative filtering over the whole interaction matrix is expensive, so Spotify runs it in batch and serves precomputed lists. This keeps request time cheap and predictable at the cost of freshness, which is fine for a weekly playlist. Fully online scoring gives instant freshness but does not scale cheaply to every user.
One Pub/Sub topic per event type versus a shared firehose
Per type topics isolate failures and let royalty critical events get their own SLO, at the cost of managing hundreds of topics and ETL clusters. A single shared stream is simpler to operate but lets a noisy event delay a business critical one, which is unacceptable for payments.
Cassandra or Bigtable versus a relational database for profiles
Per user, high write, TTL heavy attribute data fits a wide column store with tunable consistency far better than a normalized relational schema. You give up rich joins and ad hoc queries, which is acceptable because the serving path only needs fast point reads by user id.
Managed Bigtable versus self operated Cassandra
Moving to managed Bigtable removed the operational load of running many Cassandra clusters and let the team focus on product, but it ties them to Google Cloud and leaves node count as the one knob to tune, which they automated with an autoscaler. Self hosting keeps portability at a real staffing cost.
Liveness over lateness in event delivery
Delivering each event type independently in hourly buckets means one stalled type does not block the rest, which keeps the platform live. The tradeoff is that a given type can arrive late, so downstream jobs must tolerate late and out of order data rather than assuming a globally complete stream.
How Spotify actually does it
Most of this is documented by Spotify itself. Their engineering blog describes the User Profile Store built on Cassandra with a companion Entity Metadata Store, Storm topologies for real time attributes, and Hadoop Crunch jobs for batch. A later series covers the event delivery system, first on Kafka and Hadoop on premises and then migrated to Google Cloud Pub/Sub, with per event type topics, deduplication on Dataproc, encryption on Dataflow, and landing in Cloud Storage and BigQuery. The 2019 writeup reported peaks above 8 million events per second and more than 350 TB of raw events per day. Spotify also published its move from self operated Cassandra to managed Bigtable and built a Bigtable autoscaler, and it uses Scio, its Scala API over Apache Beam and Dataflow, for large scale data processing that feeds recommendations like Discover Weekly.
Sources
- Personalization at Spotify using Cassandra (Spotify Engineering)
- Spotify's Event Delivery: Life in the Cloud (Spotify Engineering)
- Why Spotify migrated its event delivery system to Google Cloud Pub/Sub (Google Cloud Blog)
- Bigtable Autoscaler: saving money and time using managed storage (Spotify Engineering)
- Big Data Processing at Spotify: The Road to Scio, Part 1 (Spotify Engineering)
Lessons to study before this interview
If any of these topics are fuzzy, the interviewer will catch it. Each lesson is 15 to 60 minutes with diagrams, code, and a quiz.