Loading...
System Design and AI Engineering, by Roni Das. 972 interactive lessons.
Start LearningAI Engineering · Intermediate · LLM and GenAI Ops
A design where more than one LLM-driven loop works on one task, each with its own context, usually coordinated by an orchestrator that hands out parts and merges the results. It adds token cost and handoffs that can fail.
In short
A multi-agent system is a design where more than one LLM-driven loop works on the same task, each with its own instructions and context, usually coordinated by an orchestrator. It helps in two narrow cases and costs extra tokens and reliability everywhere else.
In short
A multi-agent system is a design where more than one LLM-driven loop works on the same task, each with its own instructions and context, usually coordinated by an orchestrator. It helps in two narrow cases and costs extra tokens and reliability everywhere else.
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers Multi-Agent System across a run of lessons, not one page. These 4 alone are about 188 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
One agent with good tools is the right default. When a team of agents actually helps, why adding agents usually lowers reliability, and the decision rule.
An AI agent is a language model in a loop, and its reliability compounds: 95 percent per step over 20 steps is only 36 percent. Why agents fail on long tasks, and the levers that fight it.
The same questions asked two ways, answer only or think step by step and end with an Answer: line, on two small models. On 40 word problems, qwen2.5:3b went from 2 right to 37 and llama3.2:3b from 7 to 37. On two-digit sums, llama went from 23 to 37 while qwen moved from 30 to 32, a change within luck. On sorting messages, it lifted llama from 20 to 38 on a weak prompt and made no clear difference where the prompt was already good. Step-by-step replies had a median of 78 to 204 tokens, against 2 to 5 for direct ones.
Keyword search matches words; meaning search matches embeddings. Combining the two rankings (rank fusion) was measured twice. On 470 quiz questions that reuse the lessons' own words, keyword search alone found the right lesson in the top 5 for 453, meaning search with bge-m3 for 439, and both together for 455. On lesson 4's 60 questions written in other words and six languages, keyword search found 5, meaning 27, and both 22. Where keyword search matched no word at all, combining changed nothing; where it matched words in the wrong lessons, it pulled good answers down.
all part of
AI Engineering
203 lessons · about 209 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with Multi-Agent Systems, and When Not to Build Onepay once,
yours for life
An agent is a language model in a loop: it reads a goal, calls tools, reads the results and decides what to do next. A multi-agent system runs more than one of these loops on the same task. Each one has its own instructions and its own context.
The context is everything a model can see when it decides: the instructions, the history and the tool results. Each agent has its own, and that is the whole point and the whole problem.
A few more words appear in every design:
Most multi-agent designs fall into one of two shapes.
The first is the orchestrator with subagents, sometimes called orchestrator-worker. The orchestrator plans, then hands each subagent a narrow task with only the inputs it needs. Each subagent starts with a fresh context, does its part, and returns a short summary, not its full transcript. The orchestrator merges the summaries.
The orchestrator is best seen as a small state machine, not a fourth chatbot. It moves through plan, dispatch, gather and merge, then decides: is the goal met, or is another round needed? Because one component changes the state and one component decides "again or done", the run has a single spine you can trace and checkpoint. Durable execution then lets each branch be its own recorded, resumable unit.
The second is the swarm: a group of peer agents with no single leader, which read and write a shared state directly. It looks flexible. In practice it is where multi-agent systems fall apart, for reasons covered below.
The right default is one agent with a good set of tools. It has one context, one transcript, and one place where a failure happened. A team earns its place in only two cases.
If neither is true, stop at one agent. Splitting work by job titles, the way you would staff a human team, is a common mistake. People share years of background and can ask each other questions. Agents share only what you pass them.
Every extra agent adds work a single agent never did. The lesson lists five costs:
The last cost is the one people underestimate. Anthropic reported that agents used roughly four times the tokens of a normal chat, and that its multi-agent research system used about fifteen times. That is fine for a deep research report someone will pay for. For a chat feature or a routine task, it is pure loss.
Usually less. A task succeeds only if every step succeeds, so step success rates multiply. Every handoff is another step. The lesson holds the per-step success rate fixed at 0.95 and changes only the design.
Based on a figure in the lesson Multi-Agent Systems, and When Not to Build One.
A single agent with 10 steps succeeds about 60 percent of the time. A naive team does the same 10 steps plus 6 handoffs, three dispatches and three merges, and drops to about 44 percent. Splitting the work into two branches of five does not win on its own: both branches must succeed, so you still count all 14 steps and get about 49 percent.
The split wins only when the orchestrator checks each branch and reruns a failed one. A five step branch fails about 0.226 of the time. With one rerun it fails only if both tries fail, so it succeeds about 0.949 of the time. The whole task then reaches about 73 percent.
A smaller, cleaner context might also make each step more reliable than 0.95. The lesson flags that as plausible but unmeasured, so treat it as something to test, not a fact.
When several peers share mutable state and act at once, their contexts fragment and their decisions collide. Mutable just means changeable. Two agents editing the same data with no rule for who wins means one change can silently overwrite the other.
Cognition wrote this up in "Don't Build Multi-Agents". Real tasks carry many layers of nuance. When each agent sees only its slice, that nuance gets lost between them. One agent assumes something another already decided against. No single agent holds the full picture, and the fault lives in the gaps, which makes it painful to debug.
The design that survives does two things. It keeps each subagent's context isolated. And it keeps writes single-threaded: subagents return results, and only the orchestrator changes the shared result, one change at a time. Treat each subagent's summary as untrusted input and check it before merging.
Build and measure the one-agent version first. Without that baseline you cannot know whether a team is better. Then ask two questions.
Pass short structured summaries between agents, never full transcripts. Trace every model call and every handoff for a run in one place. Estimate the token cost per task before you build, and check it after. Multi-agent is one of several agentic design patterns, and usually not the first one to reach for.
To run the step-count math and the token arithmetic yourself, the Multi-Agent Systems lesson is the next step.
Anthropic's multi-agent research system
Uses an orchestrator with parallel subagents for research directions that are genuinely separate. Anthropic reported about fifteen times the tokens of a normal chat.
Cognition
Argued in "Don't Build Multi-Agents" that dividing nuanced work across agents loses context between them; its later follow-up keeps writes single-threaded.
It means more than one model-driven loop works on the same task, each with its own instructions and context. Usually an orchestrator plans the work, hands narrow parts to subagents, and merges the short summaries they return into one answer.
Because every handoff is another step that can fail, and step success rates multiply. Peers that share mutable state also lose context between them and overwrite each other's work. At 0.95 per step, a naive team doing 10 steps of work plus 6 handoffs succeeds about 44 percent of the time, against 60 percent for one agent.
No. Agentic AI covers any system where a model acts in a loop with tools, including a single agent. Multi-agent is one design pattern inside that family, where several such loops split one task. Most agentic systems work best with one agent.
Much more, because each agent fills and pays for its own context. Anthropic reported that agents used roughly four times the tokens of a normal chat and its multi-agent research system about fifteen times, so it suits only high-value work.
AI Agent
A language model placed in a loop: it decides an action, calls a tool, reads the result, and repeats until done. The hard part is that per-step errors compound over many steps.
Agentic Design Patterns
Reusable ways to structure an AI agent's work, such as ReAct, plan-and-execute, reflection and multi-agent setups. Each trades extra model calls for a chance at a better answer.
ReAct Agent
An AI agent that alternates reasoning and acting: it writes a thought, calls a tool, reads the result, and repeats until it can answer. ReAct is short for reason plus act.
Tool Calling
When a model emits a structured request to run a function, and an orchestrator executes it and feeds the result back. The model never runs anything itself, so the orchestrator validates every call.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
This page explains the idea. The "Multi-Agent Systems, and When Not to Build One" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 203 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Multi-Agent Systems, and When Not to Build One lessoncourse 2
Take models from notebook to production, with labs on real models.