A team ships an LLM feature. It demos well, the prompt looks clever, everyone is pleased. A week later someone, maybe a manager, maybe a customer, maybe the person sitting next to you, asks the one question every LLM feature eventually gets: how well does it actually work?

And the room goes quiet.

There are logs. There is a full trace of every request: the prompt that went in, the chunks that were retrieved, the tokens, the latency, the answer that came out. What there is no way to produce is a number that says the answers were good. A developer who had been building agents for over a year described it exactly: when someone asks how well the agent performs, I freeze.

That freeze is the subject of this whole chapter. It has a name. Shipping on vibes. You tune the prompt until a handful of examples look right, you ship, and from then on quality is a feeling, not a measurement. When the feature seems worse you change something, glance at a few outputs, decide it feels better, and ship again.

Watch the loop turn. Nothing in it tells you whether the system got better or worse between two ships. There is no before, no after, no evidence in either direction. Evals are the thing that breaks the cycle, because they replace feels better with a measured before and after. This lesson is about why that matters more than almost anything else you will build, and the rest of the chapter is how to do it.

Because observability answers a different question than the one you are being asked.

Observability tells you what happened. It records the trace: the input, the retrieval, the tool calls, the final output, in full detail. Evals tell you whether what happened was good. Those are not the same question, and the gap between them is where quality quietly dies.

The numbers make the gap concrete. In the LangChain State of Agent Engineering survey, which collected 1340 responses in late 2025, 89 percent of teams had observability in place, while only 52 percent ran evals. Almost everyone can see their traces. Barely half can score them.

Read the two sides as a pair. A logged wrong answer looks exactly like a logged right answer. The trace is identical, the latency is fine, no error was thrown. Observability shows you the output in perfect detail and stays completely silent on whether it was correct. That 37-point gap between watching and judging is not a small operational detail. It is the difference between a team that can improve its system on purpose and a team that can only guess.

The reason this matters right now is worth saying plainly. Model intelligence has largely commoditized. Anyone can call a strong model through an API. The thing that separates a feature that works from one that merely runs is the ability to measure its quality and improve it on evidence, which is what LLM evaluation means in practice. That is why eval literacy has become the clearest signal that someone has actually built with LLMs rather than just read about them. It is, more than prompt tricks or framework knowledge, the job.

This is the instinct every engineer brings from normal software, and it is worth understanding exactly why it does not carry over.

Classic software is deterministic. You call add(2, 2) and it must return 4, every time, forever. So a unit test asserts one exact output: assert add(2, 2) == 4. That works because there is a single correct answer and any deviation is a bug. This is the world unit tests were built for, and they are perfect at it.

LLM output is open-ended and non-deterministic. Ask a model to summarize a support ticket and a hundred different wordings can all be correct. Ask it the same question twice and you can get two different strings, both fine. There is no single ground-truth string to compare against, so output == expected has nothing to check.

Walk the fork in the diagram. The question that splits the two worlds is simple: is there exactly one correct output string? For a calculator, yes, and the unit test works. For a summary, an answer, a rewrite, a plan, no. Many outputs are valid, none is the reference, and string equality falls apart.

Evaluation is what takes the place of the unit test. Instead of matching a string, you judge a property of the output: is this summary faithful to the source, does this answer actually address the question, is this response safe. That shift, from asserting an exact value to judging a property, is the mental model the rest of this chapter rests on. If you hold onto the unit-test instinct, nothing else about evals will make sense.

They measure word overlap with a reference answer, and for open-ended text that turns out to be close to useless.

BLEU and ROUGE count how many words or short phrases the output shares with a fixed reference string. BERTScore does a softer version using embedding similarity instead of exact words. All three assume there is a canonical answer to compare against. For translation with a tight reference they had a real use once. For the open-ended generation LLMs do now, the assumption breaks, because there is no single reference. A correct answer written in different words shares few tokens with your reference and scores low. A wrong answer that happens to reuse the reference's vocabulary can score high.

You can watch this happen. The snippet below scores a correct paraphrase and a wrong-but-overlapping answer with a simple word-overlap metric, the same idea ROUGE is built on.

The correct paraphrase scores lower than the answer that reuses the reference's words while getting the meaning wrong. A metric that ranks a right answer below a wrong one is not measuring quality, it is measuring vocabulary. As Hamel Husain and others who work on evals full time put it, these scores create an illusion of confidence that is unjustified. By 2026 they are largely dead for generation work. What actually runs production evals is two things: human review and LLM-as-judge.

Stack them by trust. Human review is the anchor, the signal every other method is checked against, but it is too slow and costly to score everything. LLM-as-judge is a second model scoring against a written rubric, which scales to thousands of examples for cents. It is the workhorse, with one hard condition: it is only worth anything once you have checked that it agrees with your humans. An unvalidated judge is just another opinion. Validating it is its own lesson later in this chapter.

Usually it is bad news. A perfect pass rate almost always means your evals are too weak, not that your system is flawless.

This is the counterintuitive point that pays off across the whole chapter, so it is worth planting now. If every case passes, you are not challenging the system. Real LLM features fail in specific, findable ways: they miss a retrieved document, they ignore a constraint buried in the prompt, they invent a number, they answer a slightly different question than the one asked. An eval suite that never catches any of this is not describing a perfect system. It is describing a suite that only tests the easy path.

The move that separates people who do this well is to treat a green board with suspicion. When everything passes, the next question is not ship it, it is what failure am I not testing for. That instinct comes directly from having read your own failures, which is why the chapter starts there and not with tooling.

There is a matching trap on the other side. It is easy to write evals that check things the model already does perfectly, because those are the easy ones to write. They stay green forever and tell you nothing. Good evals cluster around the places the system actually breaks, and you only know where those are by looking.

Between the output and the next thing you fix. Evals are not a dashboard you check on Fridays. They are the feedback signal that decides where you spend effort.

Follow the arrows. An input enters the system, which is your prompt, your model, your retrieval, and any tools. The system produces an output. Observability logs that output, but cannot grade it. The eval scores it against a defined property, and here is the part that matters: the score does not just sit there. It names a specific failure mode, retrieval missed the doc, the prompt dropped a constraint, the model hallucinated, and that is what you change next.

This is the difference between an eval and a metric on a chart. A chart tells you a number moved. An eval tells you which output failed and why, so you know exactly where to put your next hour of work. When people who build this for a living say evals are the job, this loop is what they mean. The prompt tweaking, the model choice, the retrieval tuning, all of it is downstream of the signal the eval gives you. Without that signal you are back in the vibes loop from the first slide, changing things and hoping.

There is a real cost to this, and the chapter is honest about it throughout. Evals are work to build and work to maintain. A good LLM-as-judge needs a set of labeled examples behind it and ongoing upkeep as the product changes. The reason it is still worth it is that the alternative is not free either. The alternative is shipping blind, and shipping blind has a bill too. It just arrives later, as a silent quality drop nobody can explain.

In the only order that works, because each step needs the one before it.

Error analysis comes first. Before you can measure quality you have to know what failure even looks like in your product, and the only way to learn that is to read your own outputs and label how they go wrong. Hamel Husain, who writes some of the most-cited practical material on this, calls error analysis the most important activity in evals, because it tells you which evals are worth writing in the first place. Everything downstream depends on it.

Golden datasets come next. Once you know your failure modes, you turn them into a fixed set of labeled examples. This becomes the ruler every later eval is measured with, so it has to come after error analysis, not before. You cannot build a good ruler until you know what you are measuring.

LLM-as-judge comes after that. With a labeled set in hand, you automate the scoring so it runs on every change, and you learn the judge's own biases and failure modes before you lean on it.

Validating the judge comes last, because it depends on everything above. You check the judge's scores against your human labels, and only once it agrees with people is its output worth acting on. A judge you have not validated is just another opinion wearing the costume of a metric.

That is the arc. This lesson was the why. The next one, error analysis, is where the hands-on work starts, and it starts exactly where it should: with you reading real failures, not with a tool.