We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
6 system design questions lean on this idea. Each walks through the full answer.
System Design Masterclass · Messaging Event Systems
Free preview: the first 2 parts of this 8-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Messaging Event Systems
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
A teacher needs to grade 200 exam papers by tomorrow. Alone, it takes 20 hours. She recruits 10 teaching assistants, gives each 20 papers, and they all grade simultaneously. When everyone's done, she collects the graded papers and compiles the final grade sheet.
That's fan-out/fan-in. One task is split into many pieces (fan-out), distributed to parallel workers, and the results are combined into a single output (fan-in). It's the fundamental pattern for parallelizing work in distributed systems.
MapReduce is fan-out/fan-in. The Map phase fans out data to parallel workers. The Reduce phase fans in the partial results. Google built their search infrastructure on this pattern, processing petabytes of web data by splitting it across thousands of machines and combining the results. Their original MapReduce paper (2004) described processing 20 terabytes of data across 1,800 machines in under an hour.
But you don't need to be Google to use it. Image processing pipelines fan out images to resize workers and fan in thumbnails. Search engines fan out queries to index shards and fan in ranked results. Payment systems fan out risk checks to parallel fraud detectors and fan in the verdicts. Stripe processes payment events through a fan-out pipeline where different workers handle authorization, fraud detection, and ledger updates simultaneously.
Fan-out: A coordinator takes a large task and splits it into independent subtasks, distributing each to a worker via a or topic.
Fan-in: Workers publish their results to a results queue or topic. An aggregator collects all partial results and combines them into the final output.
The hard part is knowing when you're done. The aggregator needs to know how many subtasks to expect. Common approaches:
| Approach | How It Works | Tradeoff |
|---|---|---|
| Count-based | Coordinator tells aggregator "expect 10 results" | Simple, but coordinator must know count upfront |
| Timeout-based | Aggregator waits max duration, combines whatever it has | Simpler but risks incomplete results |
| Correlation ID + manifest | Coordinator publishes a manifest listing all subtask IDs | Strongest guarantee, most complex |
Real-world example, distributed search:
Without fan-out/fan-in, the search would query shards sequentially. 20x slower. Elasticsearch's coordinating node uses a count-based approach: it knows exactly how many shards to query and waits for all of them (or a configurable timeout).
The tail problem. Your fan-out is only as fast as the slowest worker. If 9 workers respond in 10ms and 1 worker takes 5 seconds, the total time is 5 seconds. Google's Jeff Dean calls this "tail at scale": at 100 workers, a 1% chance of a slow response means a 63% chance that at least one is slow. Solutions include hedged requests (send the same task to 2 workers, take whichever responds first) and speculative retries.