The 92 Percent That Meant Nothing
92 percent pass rate. That is the first number a new LLM judge reports for the support agent, and it is exactly the kind of number that ends a discussion. It goes in a slide, the feature ships wider, and the judge becomes the quality gate on every release. One question never gets asked: how good is the judge itself? A 92 percent pass rate from a judge nobody has checked is not a measurement. It is a guess with a decimal point.
Two months later a customer escalation lands on an engineer's desk. The agent had confidently given a refund policy that does not exist. She pulls the trace. The judge had marked that answer pass. She pulls a hundred more of the judge's passes and reads them by hand. Roughly one in six is wrong in a way a human reviewer would have failed on the spot. The 92 percent was real arithmetic on a judge that could not tell a good answer from a bad one.
Here is the quiet part. Nobody had ever checked whether the judge agreed with a person. They built a measuring instrument and never calibrated it, then made decisions on its readings for two months. The judge was not lying. It was just never validated, so its number had unknown accuracy, and a number with unknown accuracy is not a measurement. It is a guess wearing a percent sign.
An unvalidated judge is a confident number with unknown accuracy. A validated judge is a measurement you can trust to run at scale.
The reason this is worth a whole lesson is that the cost of skipping the check does not stay small. It compounds. The few hours you saved by not building a validation set turn into a shipped error. Then weeks of that error reaching users under a green light. Then an incident, and a loss of trust in the whole eval programme. Follow the price of one unchecked judge from the moment it grades to the moment it becomes an escalation.
This lesson closes the loop the chapter has been building toward. You have an LLM judge. Before you let it grade thousands of outputs you cannot read yourself, you have to prove it agrees with a human expert. The good news: you already know how to do this, because it is the same thing you do to any classifier.