We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Data Governance Compliance
Free preview: the first 2 parts of this 8-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Data Governance Compliance
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
A customer calls your support team. The agent needs to see their profile (from the User Service), their orders (from the Order Service), their payment history (from the Billing Service), and their support tickets (from the Ticketing Service).
Four services. Four databases. Four APIs. The agent needs a single screen showing all of it.
This is data aggregation: collecting data from multiple sources and combining it into a unified, queryable view. It powers dashboards, reports, search systems, and any feature that needs cross-service data.
In a monolith, you would write a JOIN across four tables. In microservices, those tables live in different databases on different servers. Data aggregation bridges this gap, either by querying services at runtime (API composition) or by pre-building combined views from event streams.
LinkedIn's "Who Viewed Your Profile" page aggregates data from profile, analytics, and network services into a single denormalized Espresso table. Reads come back in under 5ms at p99, even though the source data lives in three separate services.
| Strategy | How It Works | Freshness | |
|---|---|---|---|
| API Composition | Call multiple services at runtime, merge results | High (sum of all service latencies) | Real-time |
| Event-sourced aggregation | Services publish events; aggregator consumes and builds combined views | Low (pre-computed reads) | Eventually consistent (seconds to minutes) |
| Batch aggregation | Periodically pull data from all sources and build views | Low (pre-computed) | Stale (minutes to hours) |
| Change Data Capture (CDC) | Stream database changes from all sources into a combined store | Low (pre-computed) | Near real-time (seconds) |
API Composition is simplest but slowest. If three services each take 100ms, the aggregated view takes 300ms (sequential) or 100ms (parallel), plus the overhead of merging. If any service is down, the view is incomplete.
Event-sourced aggregation pre-builds the combined view. Reads are fast (one database query) but writes are eventually consistent. A new order might not appear in the aggregated view for a few seconds.
CDC is a variation of event-sourced aggregation where Debezium (or a similar tool) tails the database's write-ahead log instead of requiring each service to publish events explicitly. The source services do not even know they are being watched.
It is easy and good
“It is really nice and good Please join the course”