We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Observability Monitoring
Free preview: the first 2 parts of this 8-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Observability Monitoring
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
An engineer pulled up a trace for a slow API request. The trace showed the total duration: 400ms. But where in those 400ms was the time spent? Was it 400ms in one service, or 50ms spread across 8 services?
The answer was in the spans. Each span represents one unit of work within the trace, an handler, a database query, an external API call, a message publish. Spans have start times, durations, and parent-child relationships. Together, they decompose a trace into its individual operations, showing exactly where time is consumed.
In this case, the spans revealed: (5ms) → Auth Service (12ms) → Product Service (15ms) → Inventory Service (8ms) → Payment Gateway HTTP call (340ms) → Response assembly (20ms). The 340ms payment gateway call was the bottleneck, immediately visible in the span breakdown.
A span has the following fields:
| Field | Description | Example |
|---|---|---|
| Span ID | 64-bit unique identifier for this span | 00f067aa0ba902b7 |
| Trace ID | The trace this span belongs to | 4bf92f3577b34da6... |
| Parent Span ID | The span that initiated this span | 0123456789abcdef |
| Operation Name | What this span represents | POST /api/checkout |
| Service Name | Which service produced this span | payment-service |
| Start Time | When the operation began | 2026-04-04T10:30:12.456Z |
| Duration | How long the operation took | 342ms |
| Status | OK, ERROR, or UNSET | ERROR |
| Attributes | Key-value metadata | http.method=POST, http.status=500 |
| Events | Timestamped annotations within the span | Exception thrown at +120ms |
Parent-child relationships form a tree. The root span (no parent) represents the entry point. It spawns child spans for each downstream operation. Those children can spawn their own children.
[Root Span: POST /api/checkout] ──────────────── 400ms
├── [Auth: Verify Token] ──── 12ms
├── [Cart: Get Items] ──── 15ms
│ └── [DB: SELECT cart_items] ── 8ms
├── [Payment: Charge Card] ──────────── 340ms
│ └── [HTTP: POST stripe.com/charges] ── 335ms
└── [Response: Build JSON] ── 5ms
This tree structure is what tracing UIs render as a waterfall chart, the most intuitive visualization for understanding where time is spent in a distributed request.