Is this a video course?

No. This is an interactive, slide-based learning platform. Each lesson has rich text, animated diagrams, live code editors, and quizzes. You learn by reading, interacting, and doing, not by watching videos passively.

How long do I have access?

Forever. Both pricing tiers are one-time payments with lifetime access. This includes all current 766 lessons and any future content we add.

What level of experience do I need?

None. We start from absolute basics like 'What is latency?' and build up to distributed consensus protocols. The Foundation level assumes zero prior knowledge of system design.

How much does the system design course cost?

7.99 US dollars for lifetime access globally, or 499 Indian rupees for lifetime access in India. One-time payment, no subscription, no hidden fees. 11 lessons are free with no signup required.

What technologies are covered?

Everything from DNS and load balancers to Kubernetes, Kafka, distributed databases, consensus protocols, stream processing, security architecture, and observability. We cover principles and real-world implementations used at Netflix, Google, Amazon, Uber, Stripe, and more.

Is this useful for system design interview preparation?

Yes. The lessons are structured around the exact topics asked in system design interviews at FAANG and top-tier companies. Interactive diagrams help you practice whiteboard-style explanations. Covers everything from URL shortener design to distributed payment systems.

How is this different from ByteByteGo or Educative?

766 interactive lessons (4x more than most competitors), 16 different diagram types that build step by step, real production examples from Netflix, Google, Amazon, Uber, and Stripe, and lifetime access for a one-time payment of $7.99 instead of annual subscriptions costing 100 to 200 dollars per year.

What is the difference between pass@k and pass^k?

pass@k is the probability that at least one of k attempts succeeds, so it rises with k and measures capability. pass^k is the probability that all k attempts succeed, so it falls with k and measures consistency. A benchmark that samples many attempts and keeps the best reports something close to pass@k, while a user who relies on the agent repeatedly experiences pass^k, which is why one number can look strong while the product feels unreliable.

Why does a 95 percent per-step agent fail a 20-step task most of the time?

Because a task succeeds only if every step succeeds, so for independent steps the rates multiply rather than average: 0.95 to the 20th power is 0.358, about 36 percent. The per-step rate never changed; the length is the exponent, and the exponent is what collapses the task rate. This is the product law, and it means a per-step number you would sign off on is not a task number you can ship.

What per-step accuracy do I need for a 90 percent task success over 20 steps?

You need 99.47 percent per step, because the requirement is the inverse of the product law: p equals T to the power one over n, so 0.90 to the power one over 20 is 0.9947. That is an error budget of about 0.53 percent on every step, and missing it on any single step puts the 90 percent target out of reach regardless of the other 19 steps.

Is it better to cut steps or raise per-step accuracy?

Cut steps first, because halving the step count is exactly equal to taking the square root of the per-step rate (p to the n over 2 equals the square root of p, all to the n), and step-cutting is design work you control that also lowers cost and latency. Raising per-step accuracy hits sharply diminishing returns past about 99 percent, where you are fighting the model's tail behavior, so reserve it for steps that are genuinely irreducible.

When do retries actually improve agent reliability?

Retries help when failures are independent, meaning transient causes like timeouts, rate limits, or unlucky samples, where a retry redraws fresh odds and drives per-step failure toward its square. They do not help against systematic failures whose cause is deterministic in the input, like a schema the model consistently misreads, because the retry reproduces the same mistake. The fraction of failures that is transient sets the ceiling on what a retry policy can buy.

What does tau-bench's pass^k finding mean for production agents?

tau-bench showed that agents which pass a task once often cannot pass it eight times in a row, with pass^8 falling below 25 percent in its retail domain, because it measures the same task repeatedly and judges success by final database state rather than the transcript. The lesson for production is that a single-run benchmark score overstates the fraction of work an agent can be trusted with, and pass^k is the honest estimate of what you can put in an SLA.

A leaderboard reports an agent at 92 percent, sampling 5 attempts per task and counting the task solved if any attempt passes. You plan to ship it returning one response per user request. Which reliability will your users actually experience?

Lower, because the leaderboard measured pass@5 (capability with a verifier) while a single unverified response is closer to pass@1, The leaderboard number is a pass@k figure: it keeps the best of five attempts using the benchmark's checker to pick the winner. Shipping one response per request removes both the extra samples and the verifier, so users experience something near the single-attempt rate, which is well below the best-of-five number. Quoting a capability metric for a product that offers no way to choose among attempts is the core confusion this lesson warns about.

Your agent runs a 20-step task at 95 percent per step and completes about 36 percent of runs. You can either merge steps to cut the task to 10 steps, or invest to raise per-step accuracy to 97.5 percent while keeping 20 steps. What is true?

Both reach about 60 percent task success, because halving the step count equals a square-root upgrade in per-step accuracy, but cutting steps is usually cheaper and also lowers cost and latency, 0.95 to the 10th is 0.599 and 0.9747 to the 20th is also 0.599, because p to the n over 2 equals the square root of p, all raised to the n. The formulas tie, so the deciding factor is price: merging steps is design work you control that also cuts tokens and latency, while raising a model's per-step accuracy hits diminishing returns. Cut steps first.

You add a verifier that catches 90 percent of a step's failures and retries when it fires. Effective per-step success rises and the task rate improves, but a class of failures remains. Which class, and why does it matter?

The well-formed, plausible, wrong outputs the verifier misses; they pass silently and become trusted input downstream, so verified pipelines fail quietly rather than loudly, The fatal channel shrinks to (1 minus p) times (1 minus c), and those survivors are exactly the failures the verifier could not detect: outputs that are malformed in neither shape nor plausibility but are still wrong. They pass the check, become trusted input for later steps, and produce a confident wrong result with no exception or retry. This is why heavily verified pipelines need an end-of-run check against external ground truth.

An agent's tasks fail for two reasons in equal measure: some tool calls hit rate limits, and some misread a particular tool's schema the same way every time. You add one automatic retry per step. What happens?

The rate-limit failures shrink toward their square because they are independent, but the schema-misread failures barely move because they are systematic and reproduce on retry, Retries pay off only when the retry faces fresh odds. Rate limits are transient: the cause is not a function of the input, so a retry drives that failure rate toward its square. A consistent schema misread is systematic: the same input produces the same mistake, so the retry reproduces the failure and the budget is spent re-running a deterministic bug. Systematic failures need a change between attempts, such as a rephrase, a different model, or a fix to the contract.

Agent Reliability Math: pass@k, pass^k, and Why 95 Percent Fails

Name: System Design Masterclass
Availability: InStock

in Agents In Production

This lesson is part of the ML Platform Engineering course. Unlock the full course, a separate one-time payment, for lifetime access.

One-time payment. Lifetime access. No subscription.

Global

$20/lifetime

One-time payment via PayPal

Get Lifetime Access

5 Foundation lessons free, try before you buy

India Special

India

₹999/lifetime

One-time via UPI, cards, or net banking

Get Lifetime Access

5 Foundation lessons free, try before you buy

“The interactive diagrams alone are worth way more than the price.”

Anish K., Senior Engineer at a Bengaluru startup

Every ML Platform lesson, Foundation to Advanced

Real architecture diagrams with real tool logos

MLOps, LLMOps, and data engineering in depth

Quizzes with instant feedback

Built and reviewed by senior engineers

Lifetime access, all future content included

The first 5 Foundation lessons of this course are free, no payment needed. Try them now