We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Observability Monitoring
Free preview: the first 2 parts of this 8-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Observability Monitoring
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
A P1 alert fired at 1:47 AM. The primary on-call's phone was on silent. The secondary on-call was on a flight. The escalation policy tried the engineering manager next, who saw the notification but assumed someone else was handling it. At 3:12 AM, a customer's angry tweet prompted a VP to investigate.
The system detected the problem in 2 minutes (good MTTD). But it took 85 minutes for a human to actually start working on it (terrible MTTA). The fix itself took 10 minutes. 85 of the 97 total minutes of outage were spent waiting for someone to pick up the phone.
MTTA. Mean Time to Acknowledge, measures the gap between when an alert fires and when a human confirms they're investigating. It's the most human of the incident metrics, and it's often the weakest link in the chain.
| Factor | Fast MTTA | Slow MTTA |
|---|---|---|
| On-call schedule | Clear, well-defined, no gaps | Informal "whoever's available" |
| Notification method | Phone call with escalation | Slack message or email |
| Escalation policy | Auto-escalate after 5 min | Manual escalation only |
| Alert quality | High signal, every alert is real | Noisy. 90% are false positives |
| On-call tooling | PagerDuty/Opsgenie with mobile app | Custom scripts or manual processes |
MTTA targets:
The connection to alert fatigue: If your on-call gets 50 alerts a day and 45 are false positives, their response time for real incidents degrades. They learn to silence notifications, delay acknowledgment, and assume alerts aren't real. High MTTA is often a symptom of noisy alerting, not lazy engineers.
It is easy and good
“It is really nice and good Please join the course”