System design interview guide
Multi-Agent System Design Interview: The Reliability Math, When a Second Agent Helps, and Why Most Designs Should Be One Agent With Better Tools
A multi-agent system is the design candidates reach for when a single agent is not reliable enough, and it is usually the wrong move, because the thing making the single agent unreliable gets worse when you add more of them. The arithmetic is unforgiving and it is the first thing to put on the whiteboard. If a step succeeds 95 percent of the time, a ten-step task finishes about 60 percent of the time and a twenty-step task about 36 percent, because the probabilities multiply. Adding a second agent does not add a second chance, it adds more steps, and each handoff between agents is a fresh opportunity to lose context. There are real cases where separate agents are the right answer, and they have a specific shape: genuinely parallel subtasks over disjoint context, or a hard isolation boundary you need for security. Neither of those is the reason most multi-agent designs get proposed. This page is about telling them apart, because that judgement is what the interview is testing.
A multi-agent system is several language-model-driven loops that pass work between each other, and the interview question is almost never how to wire them up. It is whether you should. The core issue is compounding reliability: an agent that takes the right action 95 percent of the time completes a twenty-step task about 36 percent of the time, and splitting that work across agents does not reduce the number of steps, it adds coordination steps on top. Every handoff also loses information, because what one agent knows lives in its context window and what it passes on is a summary, so the receiving agent is working from a lossy copy of the situation. Those two effects are why a single agent with better tools beats a committee of agents in most real systems: replacing five reasoning steps with one well-designed tool call removes five chances to fail, and it is the highest-value move available. There are genuine exceptions and they have a recognisable shape. Parallel research over disjoint sources is one, because the subtasks do not need each other's context and the fan-out is a latency win. A hard security boundary is another, because an agent that handles untrusted input should not be the same agent that holds the credentials. Cost and latency both fan out with the number of agents, and so does the debugging surface: a failure in a five-agent system requires reconstructing five conversations to find where it went wrong, which makes tracing and durable execution structural requirements rather than nice-to-haves. The strongest interview answer starts with the arithmetic, proposes the single-agent design, names the specific conditions under which it would split, and treats the orchestration as a workflow problem with a model inside it rather than as a conversation between colleagues.
Where it shows up
Asked wherever agents are being shipped rather than demoed: the AI product teams at Anthropic, OpenAI, Google and Microsoft, the agent and automation startups, developer-tool companies building coding agents, and enterprise platform teams putting agents in front of internal systems. It appears for AI engineer, applied AI, and ML platform roles, and increasingly in senior backend interviews where the real question is whether you treat a non-deterministic component as an ordinary service. A common variant is design an AI agent that can handle customer support end to end, where the interviewer is waiting to see whether you propose five specialised agents without mentioning what happens when one of them is wrong.
Why this question is asked
Because the enthusiastic answer and the correct answer point in opposite directions, which makes it a very efficient question. Multi-agent architectures are heavily marketed and easy to draw, so a candidate who has read about them produces a clean diagram of a planner, three specialists and a critic, and never mentions reliability, context loss, cost, latency or how they would debug it. A candidate who has operated one starts with the arithmetic, points out that every box on that diagram is a chance to fail, and argues for fewer boxes. It also tests a specific engineering instinct that transfers well beyond agents: recognising when a system's problem is variance rather than capability. The reason a long agent run fails is rarely that the model cannot do any individual step. It is that doing forty steps in a row without a mistake is a different problem from doing one step well, and the fixes for it are the ordinary ones, which is to say fewer steps, checkpoints, idempotency, retries at a level that makes sense, and a human in the loop at the points where being wrong is expensive. And it rewards candidates who will say a system should be simpler, which is a harder thing to say in an interview than proposing an impressive architecture.
Requirements
Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.
Functional requirements
- Accept a goal expressed in natural language and run a loop that plans, calls tools, observes results, and decides whether the goal is met
- Call external tools with validated arguments and handle their failures, including timeouts, rate limits and malformed responses, without the loop losing the thread
- Persist run state so a run survives a process restart, a deploy, or a multi-hour wait on an external system, and resumes rather than restarting
- Decide when to stop: on success, on a step budget, on a cost budget, or on repeated failure, and report which of those happened
- Escalate to a human at defined points, with enough context for the human to act, and resume afterwards
- Where work genuinely fans out, run subtasks in parallel over disjoint context and merge the results, with a defined policy for what happens when one subtask fails
- Trace every run end to end: every prompt, tool call, result and decision, tied to one run identifier across all participating agents
- Enforce per-agent permissions, so an agent handling untrusted input cannot reach the tools that hold credentials or make irreversible changes
Non-functional requirements
- Task completion rate, measured end to end on real tasks rather than per-step accuracy, because per-step accuracy is the number that looks fine while the system fails
- Cost per completed task, not cost per call. A run that fails on step eighteen has cost eighteen steps and delivered nothing, so failures are the expensive case
- Latency: agent runs are seconds to minutes, which is a different interaction model from a request, and the system should be designed around that rather than fighting it
- Durability: a run in progress must survive an infrastructure event, because runs are long enough that infrastructure events happen during them
- Determinism where it is available: the orchestration, retries and state transitions should be deterministic even though the model is not, so failures are reproducible
- Isolation: a compromised or confused agent must be contained by permissions rather than by instructions, because instructions are exactly what an attacker manipulates
- Observability: a trace has to reconstruct the whole run, because the failure is usually three steps before the visible error
Back-of-envelope scale estimates
Show your math. Pulling numbers from thin air signals you have not thought about the load.
Compounding success over a chain of steps
0.95^10 is about 0.60, and 0.95^20 is about 0.36 (arithmetic)
The number to put on the whiteboard first. If each step is independent and succeeds with probability p, the run succeeds with p to the power of the number of steps. A 95 percent step is excellent and a 36 percent task is a broken product, and the gap between those two statements is the entire problem. It also explains why the highest-value change is usually removing steps rather than improving the model.
Steps in a real agent task
a handful for a narrow task, tens for an open-ended one (illustrative)
Step count is the exponent, so it matters more than anything else in the arithmetic above. This is the argument for collapsing several reasoning steps into one deterministic tool: a tool that does the right thing every time replaces a step whose probability is less than one, and shortens the exponent at the same time.
Context window consumed per step
grows with every tool result appended, so the later steps of a long run operate on a much fuller context (structural)
Each observation is appended to the conversation, so a long run carries an increasingly large context, which raises cost per step and degrades the model's attention to the earliest instructions. This is one reason step quality is not constant across a run, and why the independence assumption in the arithmetic above is optimistic rather than pessimistic.
Cost and latency fan-out
roughly linear in the number of agents for cost, and set by the slowest branch for latency (structural)
Every agent is its own sequence of model calls with its own context, so five agents is roughly five times the token spend for the same task unless the work is genuinely disjoint. Parallel fan-out buys latency because branches run concurrently, but the run still waits for the slowest branch, and a retry in one branch delays everything.
Human escalation rate
a design parameter, not a measurement, and it should be set deliberately (illustrative)
How often the system asks a person is a choice about where the cost of being wrong is too high to absorb. Setting it to zero is what makes an agent look impressive in a demo and dangerous in production. Treating it as a tunable, with the threshold tied to the reversibility of the action, is the production answer.
High-level architecture
Start with one agent, because that is what most designs should be, and add the second only against a stated reason. The single agent is a loop: a model receives the goal and the history, chooses a tool call, the runtime executes it, the result is appended, and the loop repeats until a stop condition. The interesting engineering is not in the loop, it is around it. The runtime that hosts the loop should be a durable workflow rather than a long-lived process. An agent run lasts seconds to minutes, sometimes longer when it waits on a human or an external system, which is long enough that deploys, restarts and instance failures happen mid-run. Modelling the run as a workflow with persisted state means each step is a checkpoint, a crash resumes rather than restarts, and retries do not re-execute side effects that already happened. That framing also gives you determinism where it is available: the orchestration, retry policy and state transitions are deterministic even though the model's output is not, so a failed run can be replayed. Tools sit behind a layer that validates arguments against a schema before execution, applies the permissions of the run, and makes destructive operations idempotent with a key derived from the run and step. Most agent failures that reach production are tool failures wearing a model costume: an argument that was almost right, a retry that duplicated a side effect, a result the model misread because it was an unstructured blob. When the system does fan out, the shape that works is an orchestrator that owns the goal and the final answer, and workers that each receive a self-contained brief and return a structured result. The orchestrator holds the context; the workers do not talk to each other. Peer-to-peer topologies where agents converse are where the context loss and the runaway loops live, and they are much harder to bound and to debug. The natural fit for this shape is genuinely parallel work over disjoint sources, where each worker's context is small and independent. The cross-cutting requirement is one trace per run, spanning every agent, every prompt, every tool call and every decision, tied to a single run identifier. Without it a five-agent failure is unresolvable, because the visible error is usually several steps downstream of the actual mistake.
In a real interview, sketch this on the whiteboard before diving into any single box.
Core components
Walk through each service. The interviewer wants to hear what each one owns, not just the names.
Agent loop runtime
Owns one run: assembles the prompt from goal, history and available tools, calls the model, parses the chosen action, executes it, appends the observation, and repeats. It enforces the budgets, which are the only thing standing between a confused model and an unbounded bill: maximum steps, maximum tokens, maximum wall clock, and maximum spend. Every one of those should terminate the run with a distinguishable reason, because which budget was hit tells you what went wrong.
Durable execution layer
Persists run state after each step so a run survives restarts and can wait hours on an external system without holding a process. This is what turns an agent from a long request into a workflow. It also provides the replay property: because orchestration is deterministic, a failed run can be re-run against the recorded model outputs to reproduce the failure, which is the difference between debugging and guessing. Temporal-style workflow engines are the common implementation and the concepts transfer whichever one you pick.
Tool layer with schema validation and idempotency
Validates every argument against a schema before execution and rejects malformed calls back to the model as a structured error it can act on, rather than letting a bad argument reach a production system. Destructive tools take an idempotency key derived from run and step, so a retry after a timeout cannot charge a card twice. Return structured results rather than prose, because a tool that returns an unformatted blob pushes parsing back into the model and adds a failure mode.
Context manager
Decides what the model sees. Naively appending every observation grows the context until cost rises, the earliest instructions lose the model's attention, and eventually the window overflows mid-run. The manager summarises or drops old observations, keeps the goal and constraints pinned, and stores the full history outside the window so a summary can be expanded when needed. This component is usually missing from whiteboard designs and is usually the reason long runs degrade.
Orchestrator
Present only when the system genuinely fans out. It owns the goal, decomposes it into briefs that are self-contained, dispatches workers, and merges structured results into the final answer. Crucially it is the only thing that holds the whole picture; workers do not have peer channels. It also owns the failure policy for branches, which must be explicit: fail the run, retry the branch, or continue with a partial result and say so.
Worker agents
Each gets a narrow brief, a small set of tools, and its own short context, and returns a structured result. Narrow scope is the point: a worker with three tools and one job has a much higher per-step success rate than a generalist with thirty, which is the only way adding agents improves rather than degrades the arithmetic. If a worker needs to ask the orchestrator a question mid-task, the brief was not self-contained, and that is a design signal rather than a feature to add.
Permission and isolation boundary
Each agent runs with its own identity and its own scoped credentials, enforced by the tool layer rather than by instructions in a prompt. This is the one architectural reason to split agents that has nothing to do with capability: an agent that reads untrusted content should not be the agent that holds write access, because prompt injection targets exactly that combination. Sandboxing for code execution belongs here too.
Trace store and evaluation harness
One trace per run covering every agent, prompt, tool call, result and decision, keyed by run identifier. It serves two jobs: debugging a specific failure, and building an evaluation set from real runs so that a prompt or model change can be measured on end-to-end task completion rather than on vibes. Per-step accuracy is the metric that looks fine while the product fails, so the harness has to score whole tasks.
Data model
Pick the right store per table. Justify each choice with the access pattern, not by reflex.
runsrun_id (primary key)goalstatus (running, succeeded, failed, escalated, budget_exceeded)step_count, token_cost, wall_clock_msterminated_by (stop, step_budget, token_budget, error, human)started_at, finished_atOne row per task, and the row a product metric is computed from. `terminated_by` is the field that makes failures diagnosable in aggregate: a system hitting its step budget constantly is a different problem from one erroring out, and a single failed status hides which. Cost is recorded per run rather than per call because cost per completed task is the metric that matters, and failed runs are the expensive ones.
stepsstep_id (primary key)run_idagent_idstep_indexaction (tool name or final answer)arguments (json)observation (json)model_versionprompt_tokens, completion_tokenscreated_atThe trace, and the single most valuable table here. `agent_id` is what lets one run span several agents while remaining one story. Storing `model_version` per step is what lets you tell a prompt regression from a model rollover later. Steps are append-only: a retry is a new step with the same index and a retry marker, never an overwrite, because overwriting destroys the evidence of the thing you are trying to debug.
agentsagent_id (primary key)rolesystem_prompt_versionallowed_tools (array)credential_scopemax_stepsAgents are configuration, not code, so a role change is a version bump rather than a deploy. `allowed_tools` and `credential_scope` are enforced by the tool layer, which is the point: an agent's permissions must not be a sentence in its prompt, because a prompt is precisely what an injected instruction can argue with.
tool_callscall_id (primary key)step_idtool_nameidempotency_keystatus (ok, invalid_arguments, timeout, error)latency_msattemptSeparated from `steps` because a single step can produce several attempts. `idempotency_key`, derived from run and step, is what makes a retry after a timeout safe on a destructive tool. The distribution of `status` across runs is the fastest way to find the tool whose schema the model keeps getting wrong, which is usually a documentation problem rather than a model problem.
escalationsescalation_id (primary key)run_idreasoncontext_snapshotassigned_toresolutionopened_at, resolved_atThe human-in-the-loop record. `context_snapshot` matters because a person cannot act on a run they cannot see, and reconstructing it from the trace at escalation time is too slow. Time to resolution is the metric that tells you whether the escalation threshold is set where your staffing can absorb it.
Deep dives
These are the conversations the interviewer is steering you toward. Practice each one until you can talk through it without notes.
The arithmetic, first, before any architecture
If a step succeeds independently with probability p, a run of n steps succeeds with p to the power n. At p = 0.95, ten steps gives about 0.60 and twenty gives about 0.36. At p = 0.99, twenty steps still only gives about 0.82. Those are not pessimistic numbers, they are what multiplication does, and they explain most of what is confusing about agent products: the individual behaviour looks excellent in a demo and the end-to-end task fails often enough to be unusable. Three conclusions follow directly, and stating them is most of a good interview answer. First, reducing n beats improving p, because n is the exponent. Replacing four reasoning steps with one deterministic tool call removes four chances to fail and shortens the chain at the same time. This is why the highest-leverage work on an agent is usually tool design rather than prompt engineering. Second, adding agents does not add attempts, it adds steps. A planner, three specialists and a critic is a longer chain than one agent doing the work, plus coordination overhead, and unless each specialist's per-step reliability is much higher than the generalist's, the product is worse. Narrow scope is what can make a specialist more reliable, so the split has to actually narrow something. Third, independence is an optimistic assumption. Errors correlate: once an agent has gone down a wrong path, subsequent steps are conditioned on that mistake and the later probabilities drop. The real curve is worse than p to the n, which is why checkpoints and verification steps, which cut the chain into shorter independent pieces, help more than the arithmetic suggests.
for p in (0.90, 0.95, 0.99):
for n in (5, 10, 20, 40):
print(f"p={p} n={n:2d} end-to-end {p ** n:.2%}")
# p=0.95 n=10 end-to-end 59.87%
# p=0.95 n=20 end-to-end 35.85%
# p=0.99 n=20 end-to-end 81.79%
#
# Read it twice: a 95% step is an excellent model and a 36% task is a
# broken product. n is the exponent, so shortening the chain beats
# improving the step, and adding agents lengthens the chain.Every handoff loses context, and the loss is invisible
What an agent knows lives in its context window: the goal, the constraints it inferred, the dead ends it already ruled out, the half-formed reason it chose this path. When it hands work to another agent it passes a message, and that message is a summary. Everything not in the summary is gone, including the things the first agent did not realise were load-bearing. This is qualitatively different from a service calling another service, where the contract is explicit and a missing field is an error. Here the contract is prose, the receiving agent cannot tell what it was not told, and it will confidently proceed on the incomplete picture. The failure shows up much later as a result that is subtly wrong rather than as an exception, which is the worst kind. Two designs mitigate it. Make briefs self-contained: a worker should receive everything it needs to do its job and nothing that requires it to ask a follow-up question, and the need to ask a follow-up is the signal that the decomposition was wrong. And keep one context holder: an orchestrator that retains the full picture and hands out narrow briefs loses far less than a chain where each agent summarises for the next and the loss compounds at every hop. The strongest version of this argument is the one that questions the split at all. If two agents need to exchange enough context to work, they are one agent with extra failure modes.
When a second agent genuinely earns its place
There are real cases, and being able to name them precisely is what separates scepticism from cynicism. Parallel work over disjoint context is the clearest. If a task decomposes into subtasks that do not need each other's intermediate state, such as researching ten sources independently or checking a claim against several systems, running them concurrently is a latency win and each worker keeps a small, focused context. The fan-out is real parallelism rather than a longer chain. The merge step is where the design work is: what happens when three of ten branches fail, and whether a partial answer is acceptable and clearly labelled as partial. A hard security boundary is the second, and it is an architectural reason rather than a capability one. An agent that reads untrusted content, a web page, an email, a user-uploaded document, should not be the agent holding write credentials, because prompt injection targets exactly that combination. Splitting them means the untrusted-input agent has no dangerous tools and the privileged agent never sees raw untrusted text, only structured output. This split is worth making even if it costs reliability, because the failure it prevents is not a wrong answer, it is an attacker acting with your permissions. A third, weaker case is genuinely different model requirements: a cheap fast model for classification and routing, an expensive one for the hard reasoning. That is really model selection rather than multi-agent architecture, and describing it as routing rather than as a team of agents is more accurate and easier to reason about. What is not on the list: giving each agent a job title. A planner, a researcher, a writer and a critic sounds like an organisation and behaves like a chain with four places to fail.
Treat the run as a workflow, not as a request
Agent runs last seconds to minutes and sometimes much longer when they wait on a human or an external system. That is long enough that deploys happen mid-run, instances are recycled mid-run, and network partitions happen mid-run. Holding the run in a process's memory means all of those lose the work, and losing the work at step eighteen means paying for eighteen steps and delivering nothing. Modelling the run as a durable workflow fixes it. State is persisted after each step, so a crash resumes from the last checkpoint. Waiting is free, because a run waiting on a human is a suspended workflow rather than a blocked thread. Retries are handled by the engine with a policy rather than by ad hoc loops. And because the orchestration is deterministic even though the model is not, a failed run can be replayed against its recorded outputs, which turns debugging from archaeology into reproduction. The part that needs care is side effects. A workflow that resumes must not re-execute a tool call that already succeeded, which is what idempotency keys are for, derived from the run and step so a retry is recognisably the same call. Get this wrong and the durability feature becomes a duplicate-charges feature. This is also where a human-in-the-loop stops being awkward. If the run is a workflow, escalating to a person is just a step that takes a long time and returns a value, rather than a special case that breaks the architecture.
Cost, latency and the debugging surface all fan out
Each agent carries its own context and makes its own model calls, so five agents on one task is roughly five times the token spend unless the work is genuinely disjoint, and the context each one accumulates is charged on every subsequent step of that agent. Cost per completed task is the metric to watch, not cost per call, because a run that fails on the last step cost everything and delivered nothing. Failure is the expensive case, which inverts the usual intuition that errors are cheap because they are short. Latency behaves differently depending on the topology. Sequential agents add their latencies; parallel agents take the slowest branch, plus the merge. Parallel fan-out is a genuine latency win and is one of the honest arguments for multi-agent designs, but a retry inside one branch delays the whole run, so branch-level timeouts and a partial-result policy are part of the design rather than an afterthought. The cost people forget is debugging. A failure in a five-agent run requires reconstructing five conversations to find which one went wrong, and the visible error is usually several steps downstream of the actual mistake. This is why one trace per run, spanning every agent and keyed by a single run identifier, is a structural requirement here in a way it is not for an ordinary service. Without it, the system is not debuggable, and a system that is not debuggable does not get better over time regardless of how good the architecture diagram looks.
Evaluate end-to-end tasks, because per-step accuracy lies
The metric that makes a multi-agent system look healthy while the product fails is per-step accuracy, and it is the one that is easiest to collect. Ninety-five percent of tool calls well-formed, ninety-five percent of reasoning steps sensible, and a third of tasks completed. Everything on the dashboard is green. The evaluation has to score whole tasks against a definition of done, on a fixed set of realistic goals drawn from actual usage, and it has to be run as a suite against prompt and model changes rather than eyeballed. Building that set from real traces is the reason the trace store earns its keep twice over. Two refinements matter. First, report the distribution of where runs fail, not just the pass rate, because a system failing consistently at step three has a fixable tool problem while one failing uniformly across steps has a capability problem, and those need different work. Second, be careful with repeated attempts: a task that succeeds if you run it five times and take the best is a different claim from one that succeeds on the first try, and conflating them is how an agent demo becomes a production disappointment. Say which one you are measuring. And hold the model version fixed while changing the prompt, and the prompt fixed while changing the model. Changing both and attributing the difference to either is the mistake that makes an evaluation worse than none.
Prompt injection is an architecture problem, not a prompt problem
An agent that reads untrusted content and also holds the ability to act is the standard vulnerable shape. A web page, an email or a document can contain instructions, the model has no reliable way to distinguish instructions it was given from instructions it read, and the consequence is an attacker steering an agent that is authenticated as your user. The defence people reach for first is a prompt that says to ignore instructions in content. It helps and it is not a control, because the attacker is writing in the same channel and can argue with it. The controls that hold are architectural. Separate the agent that ingests untrusted content from the agent that holds credentials, and let only structured, validated output cross between them. Scope every credential to the minimum the role needs, enforced at the tool layer rather than in a prompt. Require human confirmation for irreversible actions, chosen by reversibility rather than by the model's confidence. Sandbox any code execution with no network and no ambient credentials. This is the one place where splitting into multiple agents is justified on grounds that have nothing to do with reliability, and it is worth saying so explicitly in an interview, because it shows the candidate is distinguishing reasons rather than applying a blanket rule. A security boundary is a real reason to pay the coordination cost. Wanting a specialist for each job is not.
Trade-offs to discuss
Every senior interviewer expects you to surface at least 3 of these. Pick the decisions, state the alternatives, and justify your choice.
One agent with better tools versus several specialised agents
One agent keeps the context intact, keeps the chain short, and is far easier to debug and evaluate, and for most tasks it wins outright. Several agents can help when each one's narrower scope genuinely raises its per-step reliability, when the work is truly parallel, or when a security boundary demands separation. The test is whether the split narrows something real. If the specialists still need most of the same context, it is one agent wearing several hats and paying for the privilege.
Orchestrator-and-workers versus peer-to-peer agents
An orchestrator keeps one component holding the full picture and makes the run bounded, traceable and terminable. Peers that converse can in principle reach solutions no single planner would have decomposed, and in practice they lose context at every hop, are hard to bound, and can loop. Unless there is a specific reason peers must negotiate, the hub is the right default, and the cost is that the orchestrator becomes a bottleneck and a single point of failure worth designing for.
Replacing reasoning steps with deterministic tools
A tool that does the right thing every time converts a probabilistic step into a certain one and shortens the chain, which is the highest-leverage change available. The cost is engineering time and rigidity: every tool is code to build and maintain, and a narrow tool cannot handle the case nobody anticipated, which is the thing the model was good at. The usual answer is to make tools for the paths that are common and stable, and leave the long tail to the model.
Durable workflow execution versus a plain long-running process
A workflow engine gives crash recovery, free waiting, deterministic replay and a sane retry policy, all of which matter because runs are long enough for infrastructure events to occur inside them. It costs an extra system, a programming model with real constraints, and the discipline of making every side effect idempotent. For a demo it is overhead; for anything that costs money when it fails halfway it pays for itself the first time an instance is recycled.
Parallel fan-out versus sequential execution
Fan-out cuts wall-clock time when subtasks are genuinely independent, and it is one of the few honest arguments for multiple agents. It costs concurrent token spend, a merge step that has to handle partial failure explicitly, and a run that is only as fast as its slowest branch. Sequential execution is slower, cheaper per task, and much easier to reason about and to stop.
Where to put the human
Escalating often makes the system safe and slow and consumes staff time; escalating rarely makes it fast and occasionally expensive in a way that is hard to undo. Tie the threshold to reversibility rather than to the model's confidence, because confidence is not calibrated and reversibility is a property you can check. Sending drafts for approval on irreversible actions and letting everything else run is the usual shape.
Splitting agents for security rather than for capability
This split costs the same coordination overhead as any other and buys something different: containment. An agent reading untrusted input with no dangerous tools cannot be steered into doing damage, whatever it is persuaded to believe. It is worth paying for even when it makes the system slightly less capable, because the downside it prevents is an attacker acting with your credentials rather than a wrong answer.
How a Multi-Agent System actually does it
The public literature on this has converged faster than most, and usefully it has converged towards restraint. Anthropic's engineering write-up on building effective agents argues for the simplest thing that works, distinguishes workflows with predetermined paths from agents that direct themselves, and recommends starting with a single well-scoped loop rather than a multi-agent architecture. Anthropic has also published a description of a multi-agent research system, and it is worth reading alongside the first piece precisely because it describes a case where fan-out did pay: parallel research over disjoint sources, which is the shape this page argues is genuine. Cognition published a piece called Don't Build Multi-Agents, whose core point is the context-loss one above: decisions made in one agent's context are invisible to another, and the resulting conflicts are hard to see and harder to fix. OpenAI's practical guide to building agents covers the same ground from the orchestration side, including guardrails and human escalation. For the durability half, Temporal's documentation is the clearest explanation of deterministic replay and why side effects need idempotency, and the concepts carry over to any workflow engine. Our own course chapter on this is called Multi-Agent Systems, and When Not to Build One, and the neighbouring chapter on agent reliability math is where the compounding arithmetic on this page comes from. What this page deliberately does not do is quote a benchmark number for how much better a multi-agent setup performs. Those figures are extremely sensitive to the task, the harness, the number of attempts allowed and the model version, and a number quoted without all four is not evidence.
Sources
- Anthropic: Building Effective AI Agents
- Anthropic: How we built our multi-agent research system
- Cognition: Don't build multi-agents
- OpenAI: A practical guide to building agents
- Temporal: what makes workflow execution durable and replayable
- OWASP: Top 10 for Large Language Model Applications (prompt injection)
Lessons to study before this interview
If any of these topics are fuzzy, the interviewer will catch it. Each lesson is 15 to 60 minutes with diagrams, code, and a quiz.
Multi-Agent Systems, and When Not to Build One
ml-intermediate / agents in production
Agent Reliability Math: pass@k, pass^k, and Why 95 Percent Fails
ml-intermediate / agents in production
The Agent Loop: Why AI Agents Fail as Tasks Get Longer
ml-intermediate / agents in production
Durable Execution: An Agent Run Is a Workflow, Not a Request
ml-intermediate / agents in production
Agent Memory and Context: What the Model Sees, and What It Forgets
ml-intermediate / agents in production
Prompt Injection and the Lethal Trifecta: Why LLM Agents Leak Data
ml-advanced / security
Evaluating RAG and Agents: One Score Detects, Components Localize
ml-advanced / evals
Related system design interview questions
Practice these next. They lean on the same core building blocks as a Multi-Agent System.