We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Devops Cicd
Free preview: the first 2 parts of this 6-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Devops Cicd
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
A developer fixed a checkout bug where discount codes were not applied correctly. The fix worked perfectly for discount codes. But it also changed how tax was calculated on discounted items, broke the receipt PDF generation, caused the email confirmation to show wrong totals, and made the analytics event fire twice. One fix, five regressions.
The billing module had 47 test cases when the fix was written. All 47 passed. The problem was that tax calculation, receipt generation, email confirmation, and analytics had their own separate test suites that nobody re-ran after the billing change. The billing tests verified billing. They did not verify that billing's callers still got correct results.
Regression testing exists to catch this exact scenario. It re-runs existing test cases after every change to verify that previously working features still work. The test suite acts as a living specification of how the system should behave. When something breaks, the test tells you exactly which behavior changed and when.
| Strategy | How It Works | Best For | Trade-off |
|---|---|---|---|
| Full regression | Run the entire test suite | Before major releases | Slow: hours for large suites |
| Impact analysis | Only run tests affected by changed code | PR-level testing | Fast but requires dependency tracking |
| Risk-based | Prioritize tests for high-risk areas | Time-constrained testing | Misses low-risk regressions |
| Visual regression | Screenshot comparison before/after | UI changes | Catches pixel-level diffs |
Test impact analysis is the most efficient approach for CI. Tools track which tests depend on which source files. When you change billing.ts, only tests that import from billing.ts (directly or transitively) re-run.
--changedSince=main to run only tests affected by changed filesGoogle's internal test system (TAP) runs test impact analysis on every commit. Their 2016 paper reported that impact analysis reduced test execution time by 90% on average while catching 99.8% of the regressions that a full run would have caught.
Visual regression catches what functional tests miss: the button moved 3px left, the font changed weight, the modal is 2px taller on Safari. Tools like Chromatic (from the Storybook team) and Percy take screenshots of every component, diff them pixel by pixel against a baseline, and flag differences for human review.