We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningAdvanced · Reliability & Resilience
The allowed amount of unreliability derived from SLOs. If your SLO is 99.9% uptime, your error budget is 0.1% (about 43 minutes/month). Once exhausted, freeze deployments.
In short
An error budget is the amount of unreliability a service is allowed before it breaks its reliability target. It is the inverse of your Service Level Objective: if your SLO is 99.9% availability over 30 days, the error budget is the remaining 0.1%, which works out to about 43 minutes of downtime per month that you are free to spend on risk, deploys, and experiments.
In short
An error budget is the amount of unreliability a service is allowed before it breaks its reliability target. It is the inverse of your Service Level Objective: if your SLO is 99.9% availability over 30 days, the error budget is the remaining 0.1%, which works out to about 43 minutes of downtime per month that you are free to spend on risk, deploys, and experiments.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Error Budget across a run of lessons, not one page. These 4 alone are about 77 minutes of step-by-step reading, every one with a quiz.
The allowed amount of unreliability, when the budget is spent, freeze features and fix reliability.
The quantified amount of unreliability you're allowed, the most powerful tool in SRE.
The reliability targets you set for your SLIs, the line between 'acceptable' and 'we need to fix this'.
Define and measure Service Level Objectives, the practice that aligns reliability with business needs.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Error Budgetspay once,
yours for life
Every service that takes reliability seriously sets a Service Level Objective, a number like 99.9% of requests succeed or 99.95% of the time the API responds under 300ms. No real system hits 100%, and chasing 100% is wasteful, so the SLO admits that some failure is acceptable. The error budget is that accepted failure expressed as a concrete quantity.
The math is simple subtraction. Budget equals 100% minus the SLO. A 99.9% availability SLO leaves a 0.1% budget. Over a 30-day window that is 0.001 times 43,200 minutes, which is about 43 minutes of allowed downtime. Push the SLO to 99.99% and the budget shrinks to roughly 4.3 minutes per month. A request-based SLO works the same way: at 99.9% over 10 million requests, you are allowed 10,000 failed requests before the budget is gone.
The point of turning reliability into a budget is that it becomes a shared currency. Product teams who want to ship fast and SRE teams who want stability stop arguing about feelings and start spending or saving the same measurable number.
You pick a Service Level Indicator first, the raw measurement, such as the ratio of HTTP requests that return a non-5xx status, or the fraction of requests served faster than your latency threshold. You collect this from request logs, a load balancer, or a metrics system like Prometheus.
Over a rolling window, usually 28 or 30 days, you compare the SLI against the SLO. The gap between perfect and your target is the total budget. The gap between perfect and your actual performance is what you have spent. Budget remaining is the difference. A bad deploy that caused 15 minutes of 5xx errors burns 15 of your 43 minutes and the dashboard shows roughly 65% of the monthly budget still available.
Teams also watch burn rate, which is how fast the budget is being consumed relative to a steady spend. A burn rate of 1 means you will exactly exhaust the budget by the end of the window. A burn rate of 14 means a sudden incident is eating the budget 14 times faster than sustainable, and that triggers an urgent alert. Google's SRE practice popularized multi-window, multi-burn-rate alerting precisely so you page humans for fast burns and only ticket slow ones.
The headline policy is the deploy freeze. When the budget is healthy, teams ship features aggressively, run risky migrations, and experiment because failures are affordable. When the budget is exhausted, the policy flips: new feature deploys stop and engineering effort redirects to reliability work until the budget recovers in the next window. This removes the political fight over whether to slow down because the number decides.
The main trade-off is that an error budget is only as good as the SLO behind it. Set the SLO too high and you waste engineering on gold-plating users never notice. Set it too low and customers churn while your dashboard stays green. The SLI also has to reflect what users actually feel, not just what is easy to measure, otherwise you can blow the user experience while the budget looks fine.
It also requires organizational buy-in. A freeze policy that management overrides every time there is a launch deadline is theater. The budget works when leadership agrees, in advance and in writing, that running out of budget genuinely changes what the team does next.
Say a payments API commits to a 99.95% availability SLO measured monthly. That is a budget of 0.05%, about 21.6 minutes of allowed downtime in a 30-day window. The team deploys twice a day behind feature flags.
Early in the month a database migration goes wrong and the API returns errors for 8 minutes before rollback. That spends 8 of the 21.6 minutes. The dashboard now reads 63% budget remaining, and burn-rate alerts during the incident paged the on-call engineer within two minutes because the outage was burning the budget roughly 2,000 times faster than the sustainable rate.
Two weeks later a flaky third-party dependency causes intermittent failures that quietly consume another 14 minutes. The budget hits zero. The pre-agreed policy kicks in: the team pauses the new checkout feature launch, spends the rest of the window adding a circuit breaker and a retry budget around that dependency, and resumes shipping when the window resets and the budget refills.
Coined the error budget concept in its SRE practice and uses budget exhaustion to gate launches, with multi-burn-rate alerting tied to each service SLO.
Datadog
Ships SLO and error budget tracking as a product feature so teams set targets, watch remaining budget, and configure burn-rate alerts.
Grafana and Prometheus
Common open-source stack for computing SLIs from metrics and rendering error budget burn-down dashboards with recording rules.
Nobl9
A reliability platform built specifically around defining SLOs, ingesting SLIs from many sources, and enforcing error budget policies.
Subtract your SLO from 100%, then apply it to your measurement window. For a 99.9% availability SLO over 30 days, the budget is 0.1% of 43,200 minutes, which is about 43 minutes of allowed downtime. For request-based SLOs, multiply the budget percentage by total request volume to get allowed failures.
An SLA is the contractual promise to customers with penalties if you miss it. An SLO is your stricter internal target. The error budget is the leftover unreliability the SLO permits, the amount of failure you can spend before you must stop and fix things. You normally set the SLO tighter than the SLA so you have a margin.
Under a standard policy you freeze risky changes: feature deploys pause and the team redirects to reliability work like fixing flaky dependencies, adding retries, or improving monitoring. Shipping resumes when the budget recovers, typically when the rolling window moves past the bad period.
Burn rate is how fast you are consuming the error budget relative to a steady pace. A burn rate of 1 means you will exactly use the whole budget by the end of the window. A high burn rate, say 14, means an incident is eating the budget far faster than sustainable and usually triggers an immediate page.
No. Chasing 100% costs far more than it returns and ignores the fact that user-visible reliability is also capped by networks, devices, and ISPs. The error budget makes the case explicit: a non-zero budget gives you room to ship features and take risks, which is usually worth more than the last fraction of a percent of uptime.
Error Budget
The quantified amount of unreliability you're allowed, the most powerful tool in SRE
Reliability for Stochastic Systems: Error Budgets for Probabilistic Output
Uptime cannot measure a system that answers wrong while healthy. Define a quality SLO, run an error budget on the pass rate, and degrade to honest fallbacks.
SLO
Service Level Objective: a target value for an SLI, like '99.9% of requests under 300ms.' The internal engineering goal that drives reliability investment.
SLI
Service Level Indicator: a quantitative measure of service behavior, like the proportion of requests faster than 300ms. The raw metric that feeds SLOs.
SLA
Service Level Agreement: a contractual commitment between provider and customer specifying uptime, response time, and penalties for breaches. The business version of an SLO.
Chaos Engineering
Deliberately injecting failures into a system to test its resilience. Netflix's Chaos Monkey randomly kills servers to ensure the system survives.
Back Pressure
A flow control mechanism where a slow consumer signals upstream producers to slow down. Prevents systems from being overwhelmed by data they can't process.
MTTR
Mean Time To Recovery: the average time from when a failure is detected to when the service is restored. A key reliability metric that drives investment in automation and runbooks.
This page explains the idea. The "Error Budgets" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Error Budgets lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Error Budget in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
