We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningAdvanced · Reliability & Resilience
Mean Time To Detect: the average time between a failure occurring and being noticed. Shorter MTTD means better monitoring and alerting. You can't fix what you don't know is broken.
In short
MTTD (Mean Time To Detect) is the average time between when a failure starts and when your team or monitoring system first notices it. A lower MTTD means problems get caught faster, which shrinks the total time an incident hurts users.
In short
MTTD (Mean Time To Detect) is the average time between when a failure starts and when your team or monitoring system first notices it. A lower MTTD means problems get caught faster, which shrinks the total time an incident hurts users.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers MTTD across a run of lessons, not one page. These 4 alone are about 55 minutes of step-by-step reading, every one with a quiz.
How long it takes to realize something is broken, the gap between failure and awareness.
How often things break, the frequency metric that shapes your reliability investment.
How long it takes to fix the problem, from acknowledgment to all-clear.
How long until a human says 'I'm on it', the gap between alert and action.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with MTTD (Mean Time to Detect)pay once,
yours for life
MTTD is one of the four core incident metrics teams track, alongside MTTA (time to acknowledge), MTTR (time to repair or resolve), and MTBF (mean time between failures). It answers a single question: from the moment something broke, how long did it stay invisible?
The clock starts at the real onset of the problem, not when someone files a ticket. If a database started returning errors at 02:14 and an engineer first saw the alert at 02:31, the detection time for that incident is 17 minutes. MTTD is the average of those numbers across many incidents over a window like 30 or 90 days.
It matters because detection time is pure dead time. While a failure goes unnoticed, no one is acknowledging it, no one is fixing it, and users are absorbing the damage. You cannot fix what you do not know is broken, so MTTD sets a floor on how good your incident response can ever be.
Calculating MTTD requires knowing the true start time of each incident, which is the hard part. Teams reconstruct it after the fact from logs, metric graphs, and traces, then compare it to the timestamp of the first alert or first human report. Tools like PagerDuty, Datadog, and Opsgenie record these timestamps automatically so the gap can be computed without guesswork.
The lever for lowering MTTD is observability: metrics, logs, and traces feeding alerts that fire on symptoms users feel, not just on raw resource numbers. An alert on a 5 percent jump in HTTP 500 rate or a p99 latency breach detects real pain in seconds. An alert that only watches CPU usage can miss an outage entirely if the broken service is sitting idle.
Good signal-to-noise is just as important. If on-call engineers get hundreds of low-value alerts a day, they tune them out, and real failures sit in the noise for long stretches. Synthetic checks, anomaly detection, and SLO-based burn-rate alerts (popularized by Google's SRE practice) all push detection earlier without flooding the pager.
The trade-off is sensitivity versus noise. Tighter, faster thresholds cut MTTD but raise false positives and alert fatigue. Looser thresholds keep the pager quiet but let slow-burning problems hide. The goal is the lowest MTTD you can hit before false alarms start eroding trust in the alerts.
Look at MTTD when incident postmortems keep showing that the failure ran for a long time before anyone noticed. If repairs are fast once a problem is found but the discovery itself is slow, the fix is better monitoring, not a faster runbook.
MTTD is most valuable for user-facing services where minutes of silent failure translate directly into lost revenue or broken trust. For a payment API or a checkout flow, detecting an outage in 30 seconds versus 20 minutes is the difference between a blip and a headline.
It is less useful in isolation. A 10 second MTTD is worthless if it then takes 4 hours to resolve, so MTTD is read together with MTTA and MTTR to see where the real time is going. Chasing a lower MTTD also has diminishing returns once detection is faster than a human can reasonably act, at which point effort is better spent on automated remediation.
Imagine a streaming service where a recommendation service starts timing out at 21:00 on a Friday. Without symptom-based monitoring, the team only finds out at 21:40 when support tickets pile up, giving an MTTD of 40 minutes during peak traffic.
After the postmortem they add an SLO burn-rate alert on the recommendation API's error budget and a synthetic test that loads the homepage every 30 seconds. The next time the same failure happens, the synthetic check fails on the first run and the alert pages on-call within a minute. MTTD drops from 40 minutes to roughly 1 minute.
Nothing about the actual repair changed, but the total customer-facing outage shrank by nearly 39 minutes simply because the problem stopped hiding. That is the entire value of investing in detection.
PagerDuty
Timestamps incident start, acknowledgement, and resolution so teams can compute MTTD and the other incident metrics automatically.
Datadog
Detects failures through metric monitors, anomaly detection, and synthetic checks, and reports detection-time analytics across incidents.
Google SRE
Uses SLO error-budget burn-rate alerts to catch reliability problems early while keeping pager noise low, directly lowering MTTD.
Grafana
Visualizes metrics and fires alerts on symptom thresholds and anomalies, surfacing incidents before users report them.
MTTD measures how long a failure stays unnoticed, starting at onset and ending at detection. MTTR measures how long it takes to fix the problem once it is known. MTTD is about visibility; MTTR is about repair speed. Total user-facing downtime includes both, so you need to lower each one separately.
There is no universal number; it depends on the system. For critical user-facing services, mature teams aim to detect within seconds to a couple of minutes using symptom-based and synthetic monitoring. The honest target is detecting faster than your users do, so failures never get reported by a customer before your alerts fire.
For each incident, take the gap between the true failure start time (reconstructed from logs, metrics, and traces) and the timestamp of the first alert or report. Average those gaps across all incidents in a chosen window, such as the last 30 or 90 days. Incident tools record these timestamps so the average can be computed automatically.
Making alerts extremely sensitive to cut detection time generates more false positives. Too many false alarms cause alert fatigue, where on-call engineers start ignoring the pager, and a real failure ends up buried in noise. The right balance is the fastest detection you can achieve before false alarms erode trust in the alerts.
No. MTTD ends the moment the failure is detected, typically when the first alert fires. The time between detection and a human acknowledging it is MTTA (Mean Time To Acknowledge), a separate metric. Keeping them distinct shows whether your delay is in monitoring or in human response.
MTTR
Mean Time To Recovery: the average time from when a failure is detected to when the service is restored. A key reliability metric that drives investment in automation and runbooks.
Alerting
Automatically notifying engineers when metrics cross predefined thresholds. Good alerts are actionable, not noisy. PagerDuty and Opsgenie route alerts to the right on-call person.
Observability
The ability to understand a system's internal state from its external outputs. Built on three pillars: metrics, logs, and traces.
Chaos Engineering
Deliberately injecting failures into a system to test its resilience. Netflix's Chaos Monkey randomly kills servers to ensure the system survives.
Back Pressure
A flow control mechanism where a slow consumer signals upstream producers to slow down. Prevents systems from being overwhelmed by data they can't process.
SLI
Service Level Indicator: a quantitative measure of service behavior, like the proportion of requests faster than 300ms. The raw metric that feeds SLOs.
This page explains the idea. The "MTTD (Mean Time to Detect)" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the MTTD (Mean Time to Detect) lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about MTTD in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
