Pinterest System Design Interview: Visual Discovery, the Pin-Board Graph, and Recommendations at Scale
Pinterest serves visual discovery to roughly 550 million monthly active users, an estimate the company has cited publicly in recent quarters. The recommendation graph behind it is enormous: the PinSage paper describes a pin-board graph of about 3 billion nodes and 18 billion edges, and the Pixie system operates over a comparable object graph while answering 1,200 queries per second per server at a 99th percentile latency of 60 milliseconds. Pinterest has said that systems backed by Pixie drive more than 80 percent of user engagement.
Pinterest is a visual discovery engine. A pin is an image with a link and metadata, a board is a themed collection, and users save pins onto boards and follow other users and boards. The hard part is not storing pins, it is discovery: given a person or an image, find the most relevant pins out of billions in tens of milliseconds. The core data lives in heavily sharded MySQL where every object carries a 64-bit ID that encodes its shard, and all cross-object relationships are resolved with application-layer joins backed by memcache and Redis rather than SQL joins. On top of that sits a home feed built by a Smart Feed pipeline that generates candidates, scores them with a ranking model, and materializes a frozen view so the feed stays available even when a generator is slow. The recommendation core is graph-based: Pixie runs biased random walks over the pin-board graph in real time, and PinSage learns pin embeddings with a graph convolutional network so related pins can be retrieved by approximate nearest neighbor. Visual search adds a unified image embedding so a crop of an image maps to visually and semantically similar pins. Everything is read heavy, so caching and CDN image delivery matter as much as the models.
Asked at: Asked at Pinterest, Meta, and other consumer and recommendation-heavy companies, and it is a favorite general system design prompt because it blends a large graph datastore with a machine learning ranking and retrieval stack.
Why this question is asked
Interviewers like Pinterest because it forces a candidate to reason about two very different problems at once. First there is a classic sharded storage problem: how do you model pins, boards, users, and the many-to-many relationship of a pin saved onto many boards, and how do you scale that datastore without distributed joins. Second there is a retrieval and ranking problem: candidate generation, embedding-based nearest neighbor search, graph random walks, and a learned ranker, all under a tight latency budget. A strong candidate separates the read path from the write path, explains why joins are moved into the application and cache tier, and shows they understand that recommendations are a two-stage funnel of cheap recall followed by expensive precision. It rewards depth on any one subsystem while testing whether you can keep the whole thing coherent.
Requirements
Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.
Functional requirements
- Users can create pins from an uploaded or linked image and save existing pins to their boards, a repin
- A pin can be saved onto many different boards by many different users, a many-to-many relationship
- Users can create and organize boards, and follow other users and specific boards
- Users get a personalized home feed mixing pins from followed boards and users with recommended pins
- Users see Related Pins on any pin, driven by graph and embedding based recommendations
- Users can search pins by text query and by image, including a crop of part of an image, which is visual search
- Images are served in many sizes and crops across web and mobile, backed by a CDN
- Users can view a board and page through its pins in a stable order
- The system tracks engagement signals such as saves, closeups, and clickthroughs to feed ranking models
Non-functional requirements
- Very read heavy: discovery and feed reads dominate writes by a large margin
- Low latency for feed and Related Pins, on the order of tens of milliseconds for retrieval
- High availability for the home feed, which should degrade to a stale but valid view rather than error
- Horizontal scalability of the core datastore as pins and users grow
- Freshness: newly saved pins and new engagement should influence recommendations reasonably quickly
- Durability of the pin and board graph, since it is the source of truth for the product
- Cost efficiency at petabyte image scale, favoring object storage and CDN caching over origin serving
- Relevance quality measured by online A/B tests, not just offline metrics
Back-of-envelope scale estimates
Show your math. Pulling numbers from thin air signals you have not thought about the load.
Monthly active users
~550 million (estimate)
Pinterest has publicly reported MAU in this range in recent quarters. Treat as a company-reported figure that moves each quarter, not a fixed constant.
Recommendation graph size
~3B nodes, ~18B edges
The PinSage KDD 2018 paper states a pin-board graph of roughly 3 billion nodes and 18 billion edges. Pixie cites a comparable object graph of about 3 billion nodes and 17 billion edges. Published numbers.
Pins saved cumulatively
~240 billion+ (estimate)
Derived. Pinterest has referenced hundreds of billions of saved pins over the years. Exact live counts are not consistently published, so treat this as an order-of-magnitude estimate for capacity planning.
Pixie query throughput per server
1,200 QPS, p99 60 ms
Published in the Pixie paper. Each server holds the entire graph in memory and answers random-walk queries within a 60 ms 99th percentile budget, so fleet capacity scales by adding replica servers.
MySQL shard count
4,096 logical shards on 16-bit shard field
The sharding blog states they launched with 4,096 shards while the 16-bit shard field in the 64-bit ID allows up to 65,536, leaving large headroom to split without changing the ID scheme.
Read to write ratio
~100:1 or higher (estimate)
Derived. Discovery products generate many feed and Related Pins reads per save action. The exact ratio is not published, but the architecture is explicitly optimized for reads, which supports a heavily read-skewed estimate.
High-level architecture
A request enters through the edge and API gateway, which authenticates the user and routes to the relevant service. For the core object graph, services read and write sharded MySQL. Every object, a pin, board, or user, has a 64-bit ID whose top bits name the shard, so a service can compute the exact database from the ID with no lookup, connect using shard-to-host mappings kept in ZooKeeper, and fetch a single row. Because the data is sharded and joins across shards are not allowed, relationships like which pins are on a board are stored in separate mapping tables and stitched together in the application, with a memcache tier caching pin objects and a Redis tier caching board-to-pin lists. Images do not flow through this path at all: originals and their many resized crops live in object storage and are served through a CDN, so the app only stores image keys. The home feed is produced by the Smart Feed pipeline, which has a component that scores incoming pins as they are saved, a store of candidate content, and a serving layer that composes and ranks a page while falling back to a materialized frozen feed if generation is slow. Related Pins and much of discovery are served by the recommendation stack: Pixie runs biased random walks over the in-memory pin-board graph for real-time candidates, while PinSage embeddings and visual search embeddings support approximate nearest neighbor retrieval. Engagement events stream into logging and batch and streaming pipelines that retrain ranking and embedding models and refresh candidate sets.
In a real interview, sketch this on the whiteboard before diving into any single box.
Core components
Walk through each service. The interviewer wants to hear what each one owns, not just the names.
Sharded MySQL object store
The source of truth for pins, boards, users, and their relationships. Data is split across thousands of logical shards, each a MySQL database, with masters and slaves. Once data lands in a shard it never moves, which keeps ID resolution deterministic.
ID service and ZooKeeper shard map
Object IDs are 64-bit values that encode shard, type, and local ID. ZooKeeper holds the mapping from shard ranges to physical master and slave hosts, so clients discover the right database and failover promotes a slave without changing any IDs.
Cache tier: memcache and Redis
Because there are no cross-shard SQL joins, the application caches heavily. Memcache stores pin_id to pin object, and Redis stores board_id to an ordered list of pin_ids. This turns an application-layer join into a couple of fast key lookups instead of scanning shards.
Image pipeline, object storage, and CDN
On upload, the original image is stored in object storage and a set of derivative sizes and crops is generated. All variants are addressed by key and served through a CDN, so read traffic for images never touches the core datastore and origin bandwidth stays low.
Smart Feed pipeline
Builds the home feed from three cooperating roles: a worker that scores newly saved pins for a recipient, a store of scored candidate content, and a serving layer that composes, ranks, and paginates. A materialized feed provides a frozen fallback view for availability.
Recommendation engines: Pixie and PinSage
Pixie answers real-time Related Pins by running many short biased random walks from seed pins over the in-memory bipartite graph and counting visits. PinSage learns pin embeddings with a graph convolutional network so similar pins can be retrieved offline by nearest neighbor.
Visual search and ANN index
A unified image embedding maps any image or crop into a vector trained for engagement and visual similarity. An approximate nearest neighbor index over these vectors returns visually and semantically similar pins for camera search, Shop the Look, and related content.
Data model
Pick the right store per table. Justify each choice with the access pattern, not by reflex.
pins (object table, sharded)local_id (AUTO_INCREMENT primary key)pin_id (64-bit, shard+type+local)data (JSON blob)image_keylink_urlcreated_atStored as an object row with the entity serialized into a JSON blob rather than wide columns. The 64-bit pin_id encodes the shard, so the row is reachable without a global index.
boards (object table, sharded)local_idboard_id (64-bit)owner_user_iddata (JSON blob)titlecreated_atSame object pattern as pins. Board content is not stored here; the membership of pins on a board lives in a separate mapping table to preserve the many-to-many relationship.
board_has_pins (mapping table)board_id (from)pin_id (to)sequence_idadded_atIndexed on the from, to, sequence triple to page a board in order. Mappings are unidirectional, so a reverse lookup of which boards contain a pin needs its own mapping table. Hot lists are cached in Redis.
user_follows (mapping table)follower_user_id (from)target_id (to)target_type (user or board)sequence_idPowers the followed set that seeds the home feed. Kept as directional edges so building a user's inbound and outbound follow lists is a range scan on the from side within a shard.
pin_embeddings (offline, ANN index)pin_idembedding_vectormodel_versionupdated_atPinSage and visual embeddings computed in batch and loaded into an approximate nearest neighbor index. Not in MySQL; served from a dedicated vector retrieval service for Related Pins and visual search.
feed_materialized (HBase)user_idpin_idscorechunk_idgenerated_atA frozen snapshot of the user's last served home feed. Read when live generation is slow or fails, which keeps the feed available. Written at a low enough rate to stay reliable.
Deep dives
These are the conversations the interviewer is steering you toward. Practice each one until you can talk through it without notes.
The 64-bit ID and shard resolution
Every object ID packs three fields into 64 bits: 16 bits of shard ID, 10 bits of type ID, and 36 bits of local ID, with 2 reserved bits, combined as (shard << 46) | (type << 36) | local. From a pin ID alone a service can extract the shard, ask ZooKeeper which master and slave host that shard range, and issue a primary key lookup against a single database. This is the design that makes thousands of shards manageable: there is no global secondary index and no directory service round trip on the hot path. The 16-bit shard field allows 65,536 shards while they launched with 4,096, so they can split a busy shard onto new hardware without ever rewriting IDs. The tradeoff is that a shard assignment is permanent for a given object, so capacity planning and shard placement have to be right, and any operation that spans shards becomes an application concern rather than a database feature.
Application-layer joins instead of SQL joins
Once data is sharded by object, you cannot ask MySQL to join a board to its pins, because the board row and the pin rows can live on different physical databases. Pinterest models relationships explicitly with mapping tables such as board_has_pins keyed on a from ID, a to ID, and a sequence number, and resolves them in the application. To page a board, the service reads the ordered pin_ids from the mapping, often served from a Redis cache of board_id to pin_ids, then multi-gets the pin objects, often served from a memcache of pin_id to pin object. This means a single logical read becomes an ID list fetch followed by a batched object fetch, which is why the cache tier is not an optimization but a core part of correctness at latency. It also means every relationship you want to traverse in both directions needs a mapping table for each direction, since the mappings are unidirectional.
Pixie real-time random-walk recommendations
Related Pins and much of discovery are powered by Pixie, which treats the pin-board graph as a bipartite graph and computes recommendations with biased random walks. Given one or more seed pins from what the user is looking at, Pixie launches many short walks that hop pin to board to pin, and counts how often each candidate pin is visited. Frequently visited pins are the most related, and the visit-count aggregation across multiple seeds naturally weights by how connected a pin is to the user's current context. The key systems trick is that each Pixie server holds the entire graph in memory, over a billion pins and boards, so a walk is pointer chasing rather than a distributed query, which is how it hits 1,200 queries per second per server at a 60 ms 99th percentile. To keep quality high the walks are biased using pin and board features and can be steered by user-specific weights, and the graph is pruned so that overly high-degree nodes do not dominate the walks.
PinSage graph convolutional embeddings
Pixie gives fast graph proximity, but Pinterest also wants embeddings that capture both graph structure and content, so related pins can be retrieved by nearest neighbor and reused across products. PinSage is a graph convolutional network trained on the pin-board graph, described in the KDD 2018 paper on a graph of about 3 billion nodes and 18 billion edges with 7.5 billion training examples. Rather than operating on the full adjacency matrix, which is infeasible at this scale, PinSage uses short random walks to define an importance-weighted neighborhood for each node and aggregates neighbor features through learned convolutions, a method the authors call importance pooling. Training uses a curriculum of harder and harder negative examples so the model learns fine distinctions rather than only separating obviously unrelated pins. The output is a fixed-length embedding per pin that is computed in large batch jobs and loaded into an approximate nearest neighbor index, so serving a recommendation is a vector lookup rather than a model forward pass on the request path.
Visual search with a unified embedding and ANN
Visual search lets a user search with an image or a crop of an image, which is central to Pinterest because so much intent is visual. Historically each visual product had its own embedding, which was expensive to maintain and improve. The KDD 2019 work describes a single unified image embedding learned with multi-task deep metric learning that jointly optimizes for pin engagement, exact product matching, and semantic similarity, using both engagement data and human labels. At serving time an image is passed through the embedding model and the resulting vector is looked up in an approximate nearest neighbor index to return visually similar pins, which powers camera search, Shop the Look, and related content. The systems lesson here is the same as PinSage: keep the expensive learning offline and in training, and make the online path a cheap ANN retrieval, then optionally re-rank the top candidates with a heavier model. Unifying the embedding also cut operational cost, since one index and one model replaced several.
Smart Feed generation, ranking, and materialization
The home feed is not chronological. The Smart Feed pipeline separates the work into distinct roles: a worker scores each newly saved pin for its potential recipients and stores the scored candidates, and a serving layer composes a page by pulling candidates, blending followed content with recommendations, and ranking with a model such as Pinnability. The important availability idea is the materialized feed, a frozen snapshot in HBase of the feed as the user last saw it. If a content generator throws an exception or takes too long to produce a chunk, the service returns the materialized content instead of failing, so a slow ranker degrades to a slightly stale feed rather than an error page. This is a deliberate consistency-for-availability trade: the feed is allowed to be a little behind the freshest possible ordering in exchange for always returning something valid quickly, and the materialized store is written at a low enough rate that it is more reliable than the live generation path.
Image storage and CDN delivery
Images are the heaviest bytes in the system and must never be served from the core datastore. On upload the original is written to object storage and a set of derivative sizes and crops is produced for the various web and mobile surfaces, each addressed by a stable key. The MySQL object row for a pin stores only the key and metadata, not the pixels. Reads are served through a CDN, so the vast majority of image requests are satisfied at the edge and origin bandwidth stays manageable even at hundreds of billions of images. This separation also lets the image pipeline evolve independently, for example adding new crop sizes or formats, without touching the pin graph. The cost and latency argument is straightforward: object storage plus CDN is far cheaper and faster for large static blobs than any database, and it isolates a bursty, bandwidth-heavy workload from the transactional core.
Trade-offs to discuss
Every senior interviewer expects you to surface at least 3 of these. Pick the decisions, state the alternatives, and justify your choice.
Sharded MySQL with application joins versus a native graph database
The pin-board graph is a graph, but Pinterest kept core storage on sharded MySQL with mapping tables and application-layer joins because MySQL was operationally well understood and predictable at scale. The cost is that every relational traversal is hand-built and cache-dependent, but they gained deterministic ID-based routing and simple failover, and pushed the graph traversal that a graph database would do into Pixie, which is purpose built and in memory.
Permanent shard assignment versus movable data
Fixing an object to its shard for life makes ID-to-shard resolution a pure bit operation with no directory lookup, which is fast and simple. The downside is you cannot rebalance an object by moving it, so hot shards are handled by splitting shard ranges onto new hardware. They accepted reduced flexibility for a much simpler and faster lookup path.
Random walks versus learned embeddings for recommendations
Pixie random walks are fresh and need no model retraining to reflect new edges, and they run in real time. PinSage embeddings capture content and deeper structure and retrieve by nearest neighbor. Pinterest runs both because they are complementary: walks for fast, fresh graph proximity, embeddings for richer similarity and cross-product reuse, rather than betting the whole recommendation system on one technique.
Materialized frozen feed versus always generating fresh
Serving a frozen materialized feed when generation is slow trades a little freshness for strong availability, so the home page never errors out on a slow ranker. Always regenerating would be maximally fresh but fragile under load or partial failures. For a discovery feed, a slightly stale but instant feed beats a fresh feed that sometimes fails to load.
Precompute embeddings offline versus score on the request path
Both PinSage and visual embeddings are computed in batch and served as ANN lookups, so the online path is cheap and predictable. Scoring a graph convolutional model per request would be far too slow at Pinterest's traffic. The trade is that embeddings can be slightly stale between batch runs, which is acceptable because pin content changes slowly relative to how often it is viewed.
Object blobs in JSON versus fully normalized columns
Storing each entity as a JSON blob in a data column keeps the schema simple and lets the object evolve without migrations across thousands of shards. The cost is that you cannot query or index inside the blob at the database level, so all filtering and joining moves to the application and cache. Given that access is almost always by primary key ID, this is a good fit.
How Pinterest actually does it
The specifics here come from Pinterest's own engineering writeups and papers. The 64-bit ID layout, the 4,096 launch shards, the ZooKeeper shard map, the object and mapping table pattern, and the memcache plus Redis caching all come from the Sharding Pinterest blog post. Pixie's random-walk design, the 3 billion node and 17 billion edge graph, the 1,200 QPS at 60 ms figure, and the claim that Pixie-backed systems drive more than 80 percent of engagement come from the Pixie paper. PinSage's graph convolutional approach, importance pooling, curriculum of hard negatives, and the 3 billion node, 18 billion edge, 7.5 billion example scale come from the KDD 2018 paper. The unified visual embedding trained with multi-task metric learning and served through an approximate nearest neighbor index comes from the KDD 2019 visual search paper. The Smart Feed structure and the materialized HBase fallback come from the home feed engineering posts. Where a figure such as monthly active users or total pins saved is not consistently published, it is marked as an estimate.
Sources
- Sharding Pinterest: How we scaled our MySQL fleet (Pinterest Engineering)
- PinSage: Graph Convolutional Neural Networks for Web-Scale Recommender Systems (KDD 2018)
- Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time (WWW 2018)
- Learning a Unified Embedding for Visual Search at Pinterest (KDD 2019)
- Building a smarter home feed (Pinterest Engineering)
Lessons to study before this interview
If any of these topics are fuzzy, the interviewer will catch it. Each lesson is 15 to 60 minutes with diagrams, code, and a quiz.
Related system design interview questions
Practice these next. They lean on the same core building blocks as Pinterest.