Because observability answers a different question than the one you are being asked.

Observability tells you what happened. It records the trace: the input, the retrieval, the tool calls, the final output, in full detail. Evals tell you whether what happened was good. Those are not the same question, and the gap between them is where quality quietly dies.

The numbers make the gap concrete. In the LangChain State of Agent Engineering survey, which collected 1340 responses in late 2025, 89 percent of teams had observability in place, while only 52 percent ran evals. Almost everyone can see their traces. Barely half can score them.

Read the two bars as a pair. A logged wrong answer looks exactly like a logged right answer. The trace is identical, the latency is fine, no error was thrown. Observability shows you the output in perfect detail and stays completely silent on whether it was correct. That 37-point gap between watching and judging is not a small operational detail. It is the difference between a team that can improve its system on purpose and a team that can only guess.

The reason this matters right now is worth saying plainly. Model intelligence has largely commoditized. Anyone can call a strong model through an API. The thing that separates a feature that works from one that merely runs is the ability to measure its quality and improve it on evidence, which is what LLM evaluation means in practice. That is why eval literacy has become the clearest signal that someone has actually built with LLMs rather than just read about them. It is, more than prompt tricks or framework knowledge, the job.

So what is an eval, exactly, once you strip away the tooling and the jargon? It is not complicated, and getting the anatomy straight now will make every later lesson click into place. An eval is a fixed set of labeled cases, a system you run each case through, and a scorer that judges a property of every output.

The scorer is the part that surprises people coming from normal software. It does not match a string. It judges a property. A summary has no single correct wording, so you never ask does this equal the reference; you ask is this faithful to the source. Hold that distinction, because the next slide is exactly where it comes from and why it matters.

This is the instinct every engineer brings from normal software, and it is worth understanding exactly why it does not carry over.

Classic software is deterministic. You call add(2, 2) and it must return 4, every time, forever. So a unit test asserts one exact output: assert add(2, 2) == 4. That works because there is a single correct answer and any deviation is a bug. This is the world unit tests were built for, and they are perfect at it. Nobody replaces a unit test on a pure function with a fuzzy judge, and nobody should.

LLM output is open-ended and non-deterministic. Ask a model to summarize a support ticket and a hundred different wordings can all be correct. Ask it the same question twice and you can get two different strings, both fine. There is no single ground-truth string to compare against, so output == expected has nothing to check. The exact-match assertion that was your sharpest tool becomes useless the instant the output stops having one right form.

Walk the fork in the diagram. The question that splits the two worlds is simple: is there exactly one correct output string? For a calculator, yes, and the unit test works. For a summary, an answer, a rewrite, a plan, no. Many outputs are valid, none is the reference, and string equality falls apart.

Evaluation is what takes the place of the unit test on the right branch. It is not a weaker cousin of testing; it is the correct tool for open-ended output. Instead of matching a string, you judge a property of the output: is this summary faithful to the source, does this answer actually address the question, is this response safe. That shift, from asserting an exact value to judging a property, is the mental model the rest of this chapter rests on. Almost every useful language-model task lives on the right branch, which is precisely why the testing habits you carry over from normal software do not fit, and why so many teams flounder when they try to force them.

They measure word overlap with a reference answer, and for open-ended text that turns out to be close to useless.

BLEU and ROUGE count how many words or short phrases the output shares with a fixed reference string. BERTScore does a softer version using embedding similarity instead of exact words. All three assume there is a canonical answer to compare against. For translation with a tight reference they had a real use once. For the open-ended generation LLMs do now, the assumption breaks, because there is no single reference. A correct answer written in different words shares few tokens with your reference and scores low. A wrong answer that happens to reuse the reference's vocabulary can score high.

You can watch this happen. The snippet below scores a correct paraphrase and a wrong-but-overlapping answer with a simple word-overlap metric, the same idea ROUGE is built on.

The correct paraphrase scores lower than the answer that reuses the reference's words while getting the meaning wrong. A metric that ranks a right answer below a wrong one is not measuring quality, it is measuring vocabulary. As Hamel Husain and others who work on evals full time put it, these scores create an illusion of confidence that is unjustified. By 2026 they are largely dead for generation work. What actually runs production evals is two things: human review and LLM-as-judge.

Human review is the anchor, the signal every other method is checked against, but it is too slow and costly to score everything. LLM-as-judge is a second model scoring against a written rubric, which scales to thousands of examples for cents. It is the workhorse, with one hard condition: it is only worth anything once you have checked that it agrees with your humans. An unvalidated judge is just another opinion. Validating it is its own lesson later in this chapter, and it is where the trust in your whole eval setup ultimately comes from.

Usually it is bad news. A perfect pass rate almost always means your evals are too weak, not that your system is flawless.

This is the counterintuitive point that pays off across the whole chapter, so it is worth planting now. If every case passes, you are not challenging the system. Real LLM features fail in specific, findable ways: they miss a retrieved document, they ignore a constraint buried in the prompt, they invent a number, they answer a slightly different question than the one asked. An eval suite that never catches any of this is not describing a perfect system. It is describing a suite that only tests the easy path.

The move that separates people who do this well is to treat a green board with suspicion. When everything passes, the next question is not ship it, it is what failure am I not testing for. That instinct comes directly from having read your own failures, which is why the chapter starts there and not with tooling.

There is a matching trap on the other side. It is easy to write evals that check things the model already does perfectly, because those are the easy ones to write. They stay green forever and tell you nothing. Good evals cluster around the places the system actually breaks, and you only know where those are by looking. This is the entire reason the next lesson is error analysis and not a tool: reading your own outputs is what tells you which evals are worth writing in the first place, and no amount of tooling can substitute for that first look.

Between the output and the next thing you fix. Evals are not a dashboard you check on Fridays. They are the feedback signal that decides where you spend effort.

Follow the arrows. An input enters the system, which is your prompt, your model, your retrieval, and any tools. The system produces an output. Observability logs that output, but cannot grade it. The eval scores it against a defined property, and here is the part that matters: the score does not just sit there. It names a specific failure mode, retrieval missed the doc, the prompt dropped a constraint, the model hallucinated, and that is what you change next.

This is the difference between an eval and a metric on a chart. A chart tells you a number moved. An eval tells you which output failed and why, so you know exactly where to put your next hour of work. When people who build this for a living say evals are the job, this loop is what they mean. The prompt tweaking, the model choice, the retrieval tuning, all of it is downstream of the signal the eval gives you. Without that signal you are back in the vibes loop from the first slide, changing things and hoping.

Turn that signal into a working practice and you get eval-driven development, the discipline that mirrors test-driven development for a non-deterministic system. Write the eval from a real failure first, so the definition of good exists before you touch anything. Then make one change, measure the whole set, and let the number decide whether you ship or go back.

The loop is honest about cost. Writing and maintaining the eval set is real work, and a good LLM-as-judge needs a set of labeled examples behind it plus ongoing upkeep as the product changes. The payoff is that every change after that is a measured step, not a gamble, and the set only grows stronger as you feed each new production failure back into step one. One change at a time is not a nicety either: it is the only way the delta can tell you what caused what.

The honest objection to all of this is that evals are effort, and they are. So it is worth being just as honest about the alternative, because shipping blind is not free. Its bill simply arrives later, and in a currency you would rather not pay in.

Start with timing. The same regression shows up twice. It shows up first as an eval score that drops the moment you make the change, and again, much later, as the business metrics a bad product eventually moves: complaints, refunds, churn, a growing support queue.

An eval is a leading signal. Churn is a lagging one. The distance between the two dots is weeks of a worse product shipping to real people, and a green business dashboard during that window is not evidence the model is healthy. It only means the damage has not compounded far enough to show up yet. Evals let you act on the first signal, while a fix still costs a code review instead of a churned account.

Now draw the failure itself as it actually unfolds. A change ships, quality slides on every request, and because nothing errors, every operational dashboard stays green the whole time. The regression is silent, and the entire window between the change and the eventual complaint is served to real users at the lower quality.

The cost of no evals is not zero, it is deferred. You either pay a small, known cost building and running the set, or a large, unknown cost in silent regressions discovered by the people you least want to discover them. Evals move that bill to the cheapest possible moment, the commit, where a regression is a red check on a pull request instead of a lost customer. That trade, a little work now against a lot of damage later, is the whole economic case for taking evals seriously.

In the only order that works, because each step needs the one before it. Before we walk that order, it helps to see the ladder every team climbs, so you can place yourself honestly on it.

Nobody builds the top rung on day one, and you do not need to. The jump that matters most is from zero to two, from a feeling to a fixed set you can re-run, because that single move turns quality from an opinion into a number two people can argue about with evidence. Everything above it, the judge and the gate, is automation of a ruler you first have to build by hand. With that picture in mind, here is why the chapter teaches its steps in a fixed order.

Error analysis comes first. Before you can measure quality you have to know what failure even looks like in your product, and the only way to learn that is to read your own outputs and label how they go wrong. Hamel Husain, who writes some of the most-cited practical material on this, calls error analysis the most important activity in evals, because it tells you which evals are worth writing in the first place. Everything downstream depends on it.

Golden datasets come next. Once you know your failure modes, you turn them into a fixed set of labeled examples. This becomes the ruler every later eval is measured with, so it has to come after error analysis, not before. You cannot build a good ruler until you know what you are measuring.

LLM-as-judge comes after that. With a labeled set in hand, you automate the scoring so it runs on every change, and you learn the judge's own biases and failure modes before you lean on it. Validating the judge comes last, because it depends on everything above. You check the judge's scores against your human labels, and only once it agrees with people is its output worth acting on. A judge you have not validated is just another opinion wearing the costume of a metric.

That is the arc. This lesson was the why. The next one, error analysis, is where the hands-on work starts, and it starts exactly where it should: with you reading real failures, not with a tool.