We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Observability Monitoring
Free preview: the first 2 parts of this 8-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Observability Monitoring
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
A checkout system started returning errors at 2:13 AM. The on-call engineer was paged at 2:58 AM. 45 minutes later. In those 45 minutes, 8,000 customers attempted to check out and failed. At an average order value of $35, that's $280,000 in lost revenue before anyone even knew there was a problem.
MTTD. Mean Time to Detect, measures the average time between when a problem starts and when someone (or something) notices. It's the most underrated incident metric because it's the one most teams can improve the fastest.
You can't fix a problem you don't know about. And every minute between failure and detection is a minute of impact, lost revenue, frustrated users, corrupted data, accumulating silently while your team sleeps or works on other things.
MTTD is a function of your monitoring coverage, alert sensitivity, and notification speed:
| Factor | Fast Detection | Slow Detection |
|---|---|---|
| Monitoring coverage | Every critical path has alerts | Only infrastructure metrics tracked |
| Alert thresholds | Tuned based on SLOs | Generic defaults or too conservative |
| Check frequency | Every 15-30 seconds | Every 5-10 minutes |
| Synthetic monitoring | Active checks simulating user flows | Only passive monitoring of real traffic |
| Notification channel | PagerDuty with escalation | Email or Slack channel |
Common MTTD targets:
Most teams don't track MTTD explicitly, they should. It's calculated from incident data: MTTD = Average(time_detected - time_started) across all incidents. If your average MTTD is 30 minutes and you reduce it to 5 minutes, you've recovered 25 minutes of impact from every future incident.