We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningAdvanced · Reliability & Resilience
Recovery Time Objective: the maximum acceptable downtime after a disaster. An RTO of 15 minutes means the system must be back online within 15 minutes of failure.
In short
RTO (Recovery Time Objective) is the maximum amount of downtime a system can tolerate after a failure or disaster before the business is seriously harmed. An RTO of 15 minutes means the service must be fully restored within 15 minutes of going down, no matter what caused the outage.
In short
RTO (Recovery Time Objective) is the maximum amount of downtime a system can tolerate after a failure or disaster before the business is seriously harmed. An RTO of 15 minutes means the service must be fully restored within 15 minutes of going down, no matter what caused the outage.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers RTO across a run of lessons, not one page. These 3 alone are about 115 minutes of step-by-step reading, every one with a quiz.
Recovery Time Objective, how quickly you must restore service after a disaster.
Cross-region (cross data center) replication explained: sync vs async, replication lag, RPO and RTO, failover loss, and what AWS, Spanner and CockroachDB do.
How a replica is promoted to primary when a database fails, step by step. Covers RPO vs RTO, split brain, fencing, and the 2018 GitHub outage.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with RTOpay once,
yours for life
RTO is a time budget for recovery. It answers one question: once the system is down, how long do you have to get it back before the cost becomes unacceptable? That cost can be lost revenue, broken SLAs, missed regulatory deadlines, or angry customers leaving for a competitor.
RTO is a business decision dressed up as a technical number. The product and finance teams decide how much downtime they can survive, and the engineering team builds an architecture that can hit it. A payment processor might set RTO at 60 seconds. An internal HR reporting tool might happily live with an RTO of 24 hours.
RTO is almost always quoted alongside RPO (Recovery Point Objective). RTO is about time to restore service. RPO is about how much data you can afford to lose, measured backward from the moment of failure. They are different dials. You can have a tiny RTO and a large RPO, or the reverse, and the two are tuned independently.
The recovery clock starts at the moment of failure and stops when the service is verified healthy again. Every step in between eats into the budget: detecting the outage, deciding to fail over, spinning up replacement capacity, restoring or promoting a database, repointing DNS or load balancers, and running smoke tests before declaring success.
A short RTO forces you toward hot standby. You keep a fully running copy of the system in another availability zone or region, replicating data continuously, ready to take traffic in seconds. This is the most expensive option because you pay for idle duplicate capacity, but it is how you reach an RTO measured in seconds or low minutes.
A longer RTO lets you use cheaper patterns. Warm standby keeps a scaled-down copy running that you scale up on failover, giving RTO in minutes to low tens of minutes. Pilot light keeps only the core data replicated and provisions compute on demand. Backup and restore, where you rebuild from snapshots, is cheapest but can mean an RTO of hours.
RTO is not real until you test it. A documented 15-minute RTO that has never survived a game day or a chaos drill is a guess. Teams run scheduled failover exercises and measure the wall-clock recovery time, then fix whatever step blew the budget.
The core trade-off is money versus downtime. Cutting RTO from 4 hours to 5 minutes can multiply infrastructure cost several times over because you move from rebuild-on-demand to always-on redundancy. There is no point buying a 30-second RTO for a system whose users would not notice an hour of downtime at 3 AM.
Pick RTO per workload, not per company. Your checkout flow, your auth service, and your batch analytics pipeline have very different tolerance for downtime, so they deserve different RTO targets and different recovery architectures. Treating everything as tier-one wastes money; treating everything as best-effort loses customers.
A common failure mode is writing an aggressive RTO into a contract or DR plan, then never building or testing the automation to meet it. When a real incident hits, manual steps and missing runbooks blow past the number. The RTO you can prove in a drill is the only one that counts.
Say you run an e-commerce checkout that earns 50,000 dollars an hour at peak. Leadership decides losing more than five minutes of checkout is unacceptable, so they set RTO at five minutes. To hit that, you run an active-passive setup across two AWS regions with a streaming replica of the orders database in the standby region.
When the primary region fails, automated health checks detect it within about 30 seconds, a failover routine promotes the standby database, and Route 53 health-check failover repoints DNS to the standby region. Traffic is serving again in roughly three minutes, comfortably inside the budget, and a synthetic checkout test confirms the system is healthy before paging is cleared.
Contrast that with the warehouse label printer service. Reprinting labels can wait, so its RTO is four hours. There is no standby; on failure, the team restores the latest snapshot and redeploys. That saves a large redundancy bill that would have bought nothing the business needed.
Amazon Web Services
Documents RTO directly in its disaster recovery guidance, mapping the four strategies (backup-restore, pilot light, warm standby, multi-site active-active) to progressively shorter RTO targets.
Microsoft Azure
Azure Site Recovery lets teams set and test RTO for VM and workload failover between regions, reporting the measured recovery time after each drill.
Google Cloud
Publishes DR planning tiers keyed to RTO and RPO, recommending cold, warm, or hot patterns depending on how aggressive the recovery target is.
Cloudflare
Anycast and automatic failover keep DNS and edge services available so customer-facing RTO stays near zero even when individual data centers drop out.
RTO is how long you can be down before service must be restored. RPO is how much data you can afford to lose, measured backward from the failure. RTO is about recovery time; RPO is about data loss. A bank might want RTO of 1 minute and RPO of 0, meaning near-instant recovery with no lost transactions.
Effectively yes, but never literally. A true active-active multi-region setup where traffic is already served from multiple sites can survive one site failing with no visible downtime, so the practical RTO approaches zero. There is always some detection and rerouting time, so a hard zero is a marketing figure, not an engineering one.
Almost always. Shrinking RTO pushes you from rebuild-on-demand toward always-running standby capacity, which means paying for duplicate infrastructure that sits idle. The right move is to set RTO per workload so you only pay for fast recovery where the business actually needs it.
Run failover drills and chaos game days, then measure the actual wall-clock time from induced failure to verified-healthy service. If the measured time exceeds your target, find the slowest step, usually a manual or undocumented one, and automate it. An untested RTO is a hope, not a guarantee.
The business sets it, engineering builds to it. Product, finance, and risk owners weigh the cost of downtime and pick a tolerable number; engineers then choose an architecture (backup-restore, pilot light, warm standby, or active-active) that can meet it within budget.
Cross-Region Replication: Sync vs Async, Lag, RPO and the Speed of Light
Cross-region (cross data center) replication explained: sync vs async, replication lag, RPO and RTO, failover loss, and what AWS, Spanner and CockroachDB do.
Database Failover
How a replica is promoted to primary when a database fails, step by step. Covers RPO vs RTO, split brain, fencing, and the 2018 GitHub outage.
RPO
Recovery Point Objective: the maximum acceptable amount of data loss measured in time. An RPO of 1 hour means you can afford to lose up to 1 hour of data.
Disaster Recovery
A plan and set of procedures for restoring systems after a catastrophic failure. Defined by RPO (how much data you can lose) and RTO (how long you can be down).
Availability
The percentage of time a system is operational and accessible. Measured in 'nines', 99.99% availability means about 52 minutes of downtime per year.
Chaos Engineering
Deliberately injecting failures into a system to test its resilience. Netflix's Chaos Monkey randomly kills servers to ensure the system survives.
Back Pressure
A flow control mechanism where a slow consumer signals upstream producers to slow down. Prevents systems from being overwhelmed by data they can't process.
SLI
Service Level Indicator: a quantitative measure of service behavior, like the proportion of requests faster than 300ms. The raw metric that feeds SLOs.
This page explains the idea. The "RTO" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the RTO lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about RTO in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
