Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningAI Engineering · Advanced · LLM and GenAI Ops
Reading real failure traces and coding them into categories to find where a system actually breaks, before building any metric or dashboard. Most of eval effort should go here, not into tooling.
In short
Error analysis is reading a sample of your system's real outputs one at a time, writing a plain note on what went wrong in each, then grouping those notes into named, counted failure types. The ranked list tells you what to fix first and which checks are worth building.
In short
Error analysis is reading a sample of your system's real outputs one at a time, writing a plain note on what went wrong in each, then grouping those notes into named, counted failure types. The ranked list tells you what to fix first and which checks are worth building.
Learn it in the course
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers Error Analysis across a run of lessons, not one page. These 4 alone are about 208 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
How to read real LLM failures before you build metrics. Open coding, axial coding and saturation turn notes into a ranked list of failure types.
Teams abandon error analysis in a text editor and record it as a conclusion about the method. Three behaviours fix it, one feature ruins the measurement, and the whole thing is a hundred lines of Python.
A measured lab where the score moved two points and the diagnosis flipped completely. Ten wrong answers before, ten after, and opposite work to do. The order you fix things in decides whether a sprint lands on the broken stage or the working one.
You will read a few dozen traces out of a hundred thousand. Measured on the lab: the same 40 traces report a failure rate of 25 percent, 100 percent or 86 percent, and the only thing that changed was the rule that picked them.
all part of
AI Engineering
210 lessons · about 219 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with Error Analysis: Reading Failures Before Building Metricspay once,
yours for life
A trace is the full record of one request: what the user sent, what was retrieved, every tool call, and the final reply. Error analysis means reading a sample of real traces, one at a time, and writing down what went wrong in each. You do this before you choose any category, score or dashboard. The output is a ranked list of named failure modes, meaning the specific ways your system breaks and how often each one happens.
The reason to start here is that your guesses about how your own system fails are usually wrong. You built it, so you have a mental picture of its weak points, and that picture hides the failures you did not imagine. If you build metrics first, you measure the failures you guessed, with great precision, and stay blind to the ones that are really happening.
Picture a team whose dashboard shows an average helpfulness of 4.2 out of 5, scored by a judge model, while customers keep escalating. They have thousands of traces and nobody has read them. Hamel Husain, who teaches evals to teams, estimates that 60 to 80 percent of real eval work is this reading and analysis. That split is one practitioner's estimate, not a measurement, but the shape matters: the tooling is the smaller part.
The first pass is open coding. You read one trace and write one short note in plain words, the way you would message a teammate: 'grabbed the billing doc instead of the returns doc', or 'correct, but three paragraphs for a yes-or-no question'. The one rule is that you use no categories yet. You describe; you do not sort. A vague note like 'bad answer' helps nobody, while 'cited a refund policy deleted last quarter' names something you can count and later check.
Read the whole exchange, not only the final answer, and pin each problem to the step where it happened. One trace often holds more than one failure. A retrieval failure and a prompt failure both look like a wrong answer from the outside, but you fix them in different places.
The second pass is axial coding. You read all your notes back, group the ones that describe the same underlying cause, give each group a short name, and count it. The categories come out of the notes, not from a list you prepared. Keep the two passes apart: if you start sorting while you read, your first three groups quietly become a checklist and every later trace gets forced into one. Both techniques come from grounded theory, a qualitative research method from the late 1960s, and were adapted for LLM evaluation by Hamel Husain and Shreya Shankar.
You stop when reading more traces stops producing new failure types. This is called theoretical saturation. Early traces keep revealing new codes, then new ones arrive more slowly, and finally a long run of traces adds nothing new.
In one measured run of 40 traces, only 8 of them added a code that had not been named before, and the last new code arrived at trace 31. The nine traces after it added nothing, which was the signal to stop. Thirteen codes from 40 traces was that run's result; the shape of the curve is what carries over to other systems, not the count.
Saturation only means something if the sample is fair. If most of your traces come from one easy request type or one quiet hour, the curve flattens early and misleads you, because you have covered a corner of the product, not the whole of it. Sample across time, request types and user groups. It also helps to have a second person code the same batch: where two careful readers pick different codes, the definition is unclear, so decide once in writing which code wins and add a worked example.
A triage agent handled 12 support tickets, each with 6 decisions to get right, such as the order id, the amount and whether the item was inside the returns window. Each ticket ran 5 times, so there were 360 calls, on a model with four billion parameters running on a laptop. There were 81 step failures, and they reduced to only 13 distinct combinations of step, answer given and answer expected, few enough to read every one.
Coding them gave four names. 'Hedged to false', a judgement that needed a comparison answered with the safe-looking 'no', covered 44 failures (54 percent). 'Wrong downstream decision' covered 23 (28 percent), 'category confusion' 10 (12 percent), and 'null written as zero' 4 (5 percent). The task-level pass rate for the run was 21.7 percent. That number was true and measured, and it gave no next step. The coding pointed at one habit in one kind of question, which points at one prompt.
Reading also found something no dashboard would plot. Each decision was a separate call that saw only the ticket, yet the final decision was wrong 57 percent of the time when an earlier premise, judged in a different call, was wrong, and 20 percent of the time when that premise was right. Both were wrong together 17 times, against 11.5 expected if the two were independent. Nothing leaked between calls: the model re-derived the same judgement and got it wrong the same way.
When you label a trace, use pass or fail on one named failure, not a 1 to 5 score. Two people rating the same answer on a 1 to 5 scale disagree often, especially in the middle, so the average is mostly noise. A sharp yes-or-no question, such as 'did it answer in the language the user wrote in?', gets far more agreement.
The tool you need is a simple way to look. A spreadsheet with one row per trace and a column for your note is enough to start. When that gets slow, build a small view that shows the whole exchange, a box for a free note, and a button per code. If reading is painful, people read ten traces and stop.
The first read cannot be handed to a model, because you do not yet know what to tell it to look for. Reading 100 traces is roughly half a day of focused work. After that it gets cheaper: each named failure becomes a yes-or-no check, many of those become simple rules like a length or language check, and a judge model is saved for things a rule cannot see, such as faithfulness. You go back to reading when the product changes or a new kind of failure slips past the checks.
LangSmith annotation queues
LangSmith lets a team send traced runs to a review queue where people read each one and attach feedback or labels.
Arize Phoenix
An open-source tool for tracing LLM applications that shows each trace step by step and lets you annotate spans and run evaluations on them.
Label Studio
An open-source data labelling tool that can be set up to show a text record and collect a label or free-text note for each one.
pandas
The Python data library is often used for the counting step, turning a table of coded traces into a ranked count per failure type with value_counts.
It is reading a sample of your system's real outputs one at a time and writing down what went wrong, before you choose any metric. The notes are then grouped into named failure types and counted. The ranked list tells you what to fix first and which automatic checks are worth writing.
Open coding is the first pass: one plain note per trace, with no categories. Axial coding is the second pass: you group the notes by shared cause and name each group. Keeping them apart stops early guesses from becoming a checklist that hides new failures.
Read until new traces stop producing new failure types, which is called theoretical saturation. In one measured run of 40 traces, the last new code appeared at trace 31. Make sure the sample covers different times, request types and users, or it will saturate early on an easy slice.
People disagree a lot on the middle of a 1 to 5 scale, so averages of those scores are mostly noise and cannot safely gate a release. A yes-or-no question about one named failure gets much more agreement, and you can measure that agreement between two readers.
Not the first pass. The point of the first read is to discover failure types you did not expect, and you cannot tell a model what to look for before you know. Once the types are named, a model can apply them at scale, after you check its labels against human ones.
Building an Annotation Tool: A Hundred Lines That Decide Whether Anybody Finishes
Teams abandon error analysis in a text editor and record it as a conclusion about the method. Three behaviours fix it, one feature ruins the measurement, and the whole thing is a hundred lines of Python.
The Error Hierarchy: Which Failure to Fix First
A measured lab where the score moved two points and the diagnosis flipped completely. Ten wrong answers before, ten after, and opposite work to do. The order you fix things in decides whether a sprint lands on the broken stage or the working one.
Which Traces to Read: Sampling Decides What You Can Discover
You will read a few dozen traces out of a hundred thousand. Measured on the lab: the same 40 traces report a failure rate of 25 percent, 100 percent or 86 percent, and the only thing that changed was the rule that picked them.
Open Coding: Writing Down What Went Wrong, Before You Have Categories
The messy first pass where you describe failures in your own words. One grader wrote thirteen different codes across ten failed traces, and seven of them mean the same thing. That mess is the point.
Axial Coding: Turning Thirteen Words Into Four Categories
Merging near-duplicate codes is not tidying. On the lab's real data it changed which failure was the most common, because the biggest category had been written seven different ways.
LLM Evaluation
Measuring whether a model or pipeline actually works, using error analysis, labeled datasets, and validated judges rather than a glance. The single most in-demand and least-taught AI engineering skill.
LLM-as-Judge
Using a language model to score another model's output. Fast and scalable, but it has real failure modes such as self-preference and position bias, so the judge itself must be validated against human labels.
Golden Dataset
A curated set of inputs with known-correct expected outputs, used to measure quality and catch regressions when you change a prompt, a model, or the retrieval stack.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Agentic RAG
Agentic RAG is RAG where a language model runs the search in a loop. It decides whether to search, what to search for and where. Then it checks if the text it found answers the question.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
This page explains the idea. The "Error Analysis: Reading Failures Before Building Metrics" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 210 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Error Analysis: Reading Failures Before Building Metrics lessoncourse 2
Take models from notebook to production, with labs on real models.