We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Observability Monitoring
Free preview: the first 2 parts of this 7-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Observability Monitoring
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
A database team detected a failure at 11:00 PM. They acknowledged the alert at 11:03 PM. They spent the next three hours trying to understand the root cause, checking logs, reviewing recent changes, running diagnostic queries, consulting documentation. At 2:05 AM, they found it: a configuration change from that morning had set the replication batch size too high. They reverted the config. Replication resumed in 2 minutes.
Total MTTR: 3 hours and 4 minutes. Time to fix once they understood the problem: 2 minutes. The fix was trivial, the diagnosis was the bottleneck.
MTTR measures the time from when you start working on an incident to when it's fully resolved. It's the metric that tells you how quickly your team can go from "we know something is broken" to "it's fixed and users are happy again."
MTTR breaks down into distinct phases:
| Phase | Activity | Typical % of MTTR |
|---|---|---|
| Triage | Assess severity, identify affected systems | 5-10% |
| Diagnosis | Understand what's wrong and why | 40-60% |
| Remediation | Apply the fix | 10-20% |
| Verification | Confirm the fix worked, monitor for recurrence | 10-20% |
Diagnosis dominates. The actual fix, reverting a deployment, scaling up instances, patching a config, is almost always fast. The hard part is figuring out what to fix.
This is why observability investments (better logs, metrics, traces) have an outsized impact on MTTR. They don't help you fix things faster, they help you find the problem faster. And finding the problem is 50% of the work.
Industry MTTR benchmarks: