We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Reliability Resilience
Free preview: the first 2 parts of this 7-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Reliability Resilience
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
A SEV1 fires at 2:47 AM. Within 4 minutes, someone's phone rings. The question is: whose phone? The answer lives in the on-call schedule, and getting that answer wrong is how real outages turn into multi-hour disasters.
On-call schedules look simple on the surface. A calendar with names and dates. But production schedule management at scale is a surprisingly deep problem. PagerDuty processes schedules for hundreds of thousands of teams, and their most requested feature category year after year is schedule management. Not alerting. Not dashboards. Schedules. Because schedules are where human life collides with operational responsibility. Vacations, sick days, religious holidays, custody arrangements, time zones, daylight saving transitions. Every edge case is a potential coverage gap, and a coverage gap at the wrong moment is a missed page that costs real money.
Google's SRE team learned this the hard way and documented the lessons. Their published guidance recommends no more than 2 on-call shifts per quarter per engineer. Exceed that and attrition climbs. Miss a schedule gap and the incident goes unowned. The schedule is the most human part of the incident response system, and that is exactly what makes it the hardest to get right.
An on-call schedule answers one question for every minute of every day: who is responsible right now? The answer feeds into the escalation policy, which feeds into the routing engine, which feeds into the phone call that wakes someone up. If the schedule has a gap, no phone rings. If the schedule points at the wrong person, the wrong phone rings.
| Component | What It Does |
|---|---|
| Base rotation | Assigns each team member to a repeating shift pattern (usually weekly) |
| Weekend layer | Overrides the base rotation with shorter Saturday/Sunday shifts |
| Holiday layer | Handles regional and company holidays with dedicated coverage |
| Override layer | Manual swaps for vacations, sick days, and ad-hoc changes |
| Schedule engine | Evaluates all layers for each minute, resolves conflicts, returns one name |
| Gap detection | Nightly scan for uncovered minutes in the next 30 days |
The layered model is the industry standard. PagerDuty, Opsgenie, and Splunk On-Call all implement it the same way because the alternative (flat schedule with manual edits) breaks at any team larger than 4 people.