Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningAI Engineering · Advanced · LLM and GenAI Ops
Using a language model to score another model's output. Fast and scalable, but it has real failure modes such as self-preference and position bias, so the judge itself must be validated against human labels.
In short
LLM-as-judge means using one language model to grade another model's output against a rubric, so you can score thousands of answers without people reading each one. It is fast and cheap, but it has measurable biases, so a judge must be tested against human labels before its scores are trusted.
In short
LLM-as-judge means using one language model to grade another model's output against a rubric, so you can score thousands of answers without people reading each one. It is fast and cheap, but it has measurable biases, so a judge must be tested against human labels before its scores are trusted.
Learn it in the course
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers LLM-as-Judge across a run of lessons, not one page. These 4 alone are about 247 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
The full catalogue of ways an LLM judge fails, the mechanism behind each one, the fix that removes it, and how to validate a judge against human labels before you trust its scores.
An LLM judge is worthless until you know how well it agrees with a human expert. Validate it like a classifier: TPR, TNR, kappa, and the human ceiling.
Two graders on the same 40 traces, two rubrics, and the SAME raw agreement of 0.93 both times. Kappa says 0.69 and 0.82, because raw agreement cannot see how much of the matching was luck.
Three cheap judges, measured on the same 40 traces. One scored 90% accuracy while missing nothing, and 82% while missing three failures in seven. A judge is a classifier, a failure is the positive case, and the two error rates cost different people different things.
all part of
AI Engineering
210 lessons · about 219 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with LLM-as-Judge and Its Failure Modespay once,
yours for life
An LLM judge is a language model given a rubric (written grading instructions) and asked to grade another model's answer, either alone (pass or fail, or a score) or against a second answer (which is better, A or B). It exists because the older options both fall short on open-ended output. People grading by hand is accurate but does not scale to every answer of every release.
Word-overlap metrics such as BLEU and ROUGE score text by how many words it shares with a reference answer. That fails when there is no single right answer, such as a summary or a support reply. A good answer in different words scores badly, and a poor answer that copies the reference scores well.
So teams use a model as the grader. In LangChain's 2025 State of Agent Engineering survey, 53.3 percent of teams running agents in production said they use LLM-as-judge, second only to human review at 59.8 percent. The risk is that a judge returns a clean number every time, and a clean number reads as truth even when it is measuring something other than quality.
Position bias means the verdict depends on which answer is shown first. In one lab, a 4-billion-parameter model running locally at temperature 0 judged 30 pairs in both orders. On 20 pairs with a clearly better answer, 19 verdicts survived the swap. On 10 pairs built as genuine ties, 6 flipped when only the order changed. Counting how often it picked slot A gave 31 of 60, 51.7 percent, which looks perfectly fair. Only running both orders and counting flips found the problem.
Sycophancy means the judge leans toward whatever the prompt hints you want. In another lab with qwen3:4b as the judge on 30 tie pairs in both orders, re-sending the same prompt changed 0 of 60 verdicts. Adding one neutral line saying which answer came from the system under test changed 24 of 60. A placebo sentence of the same length that named neither answer changed 4. A line stating an opinion changed all 60.
Other documented biases include verbosity (preferring longer answers), formatting (preferring bullet points and headings over the same facts in a paragraph), and self-preference, where a judge scores output from its own model family higher; Panickssery and others showed at NeurIPS 2024 that models can recognise and favour their own text. Documented does not mean universal: a lab that padded answers to 2.3 times their length with no new facts did not find the length bias in its small local judge. Test each bias on your own judge.
A judge can change its verdict when nothing important changed. In one lab, six reasonable phrasings of the same rubric, on the same 60 answers, moved the pass rate by 11.7 points. In another, running the same eval six times on the same answers with the same prompt moved the result by 6.7 points. A release gate set tighter than that noise will fire on its own.
The format of the question matters more than the size of the judge. A 1-to-5 scale asks the judge to hold distinctions it cannot keep steady, so the same answer drifts between a 3 and a 4. A binary pass or fail is steadier, and a pairwise comparison steadier still. DoorDash reported that splitting a fuzzy relevance question into concrete yes-or-no checks, such as whether the result matches the query intent and whether the item is available, worked better than reaching for a smarter judge.
Order the output so reasoning comes before the verdict. When a judge writes the number first, the reasoning is a story told to fit a snap decision. Writing reasoning per criterion first and the verdict last, the idea behind judges such as G-Eval, costs a few tokens and gives you a rationale you can inspect.
The only thing that tells you a judge is trustworthy is measuring how often it agrees with careful human labels on the same work. Treat the judge as a detector, like a smoke detector, where the thing it looks for is a failure. Then two numbers matter: the true positive rate (of the answers that really failed, the share the judge flagged) and the true negative rate (of the answers that were fine, the share it passed). Plain accuracy mixes the two and can hide a judge that misses most failures.
The labels you check against matter too. In one lab, the same judge on the same 40 traces caught 100 percent of failures when checked against one careful grader and 57 percent against another. Three traces caused the whole gap. A judge's accuracy is always accuracy against a particular set of labels, so write down whose labels they are and how they were made.
Confidence is not accuracy either. Judges tend to be overconfident, especially at the top of their scale. Build a calibration curve from a labelled sample, showing how often grades at each confidence level match people, and read future scores through it before setting any pass threshold.
Run every pairwise comparison in both orders and score a flip as a tie. Use a judge from a different model family than the system under test. Strip anything from the judge prompt that is not the question, the answers and the rubric: model names, which answer is the new version, and anyone's opinion. Pin the judge prompt and the judge model version in version control, and re-validate when either changes.
Keep the eval judge out of any training reward loop. A model trained to please a fixed judge learns the judge's biases faster than it learns quality, and the score rises while the product does not. This is Goodhart's law: a measure that becomes a target stops being a good measure.
For close calls, use a panel of judges from different families. Automate where they agree, and send cases where they split to a person. A panel is more reliable than one judge, but agreement among biased judges is still not the same as being right, so the panel also needs checking against human labels.
MT-Bench
A benchmark and study (Zheng et al., 2023) that popularised using strong models as judges and measured position and verbosity bias in them.
AlpacaEval (length-controlled)
An automatic evaluator whose length-controlled version adjusts for judges' preference for longer answers.
G-Eval
A method that has the judge reason through evaluation steps before giving a score, used to improve agreement with human ratings.
Ragas
An open-source library for evaluating retrieval-augmented generation that uses LLM judges for metrics such as faithfulness to the retrieved context.
Only after you measure it. Judges have position bias, sensitivity to prompt wording, and run-to-run noise; in one lab, one neutral line in the prompt changed 24 of 60 verdicts. Compare the judge with careful human labels on the same work, using true positive and true negative rates, before you trust its scores.
It is when the verdict depends on which answer is shown first. The check is to run each pair in both orders and count how often the winner flips. In one lab, 6 of 10 genuine ties flipped, even though the judge picked slot A in only 51.7 percent of calls overall.
Pass or fail, or a pairwise comparison, is usually steadier than a fine scale, because the judge cannot hold small distinctions steady across runs. Splitting a fuzzy question into several concrete yes-or-no checks makes it steadier still and shows which check failed.
It should not be the only judge. Research at NeurIPS 2024 found models can recognise and favour their own text. Use a judge from a different model family, or measure the same-family judge against human labels on that model's output.
Sample real outputs, weighted toward hard cases, and have people label them with the same rubric. Run the judge on the same sample and measure how many real failures it catches and how many good answers it passes. Record whose labels you used, pin the judge prompt and model, and repeat when either changes.
Validating Your Judge: TPR, TNR, and the Human Ceiling
An LLM judge is worthless until you know how well it agrees with a human expert. Validate it like a classifier: TPR, TNR, kappa, and the human ceiling.
Inter-Annotator Agreement: Before You Trust Your Own Labels
Two graders on the same 40 traces, two rubrics, and the SAME raw agreement of 0.93 both times. Kappa says 0.69 and 0.82, because raw agreement cannot see how much of the matching was luck.
What a Judge Score Actually Means
Three cheap judges, measured on the same 40 traces. One scored 90% accuracy while missing nothing, and 82% while missing three failures in seven. A judge is a classifier, a failure is the positive case, and the two error rates cost different people different things.
Which Labels You Check Against
The same judge, the same 40 traces, checked against two careful graders instead of one. It caught 100% of failures against the first and 57% against the second. Three traces caused the whole gap, and both graders agreed about all three.
Does Giving the Judge a Third Box Help?
The last lesson found our judge was scored on a choice it was never allowed to make. So we gave it the third box the graders had, and ran the same 40 traces again. It got worse in five of the six judge-and-grader pairs, and the reason is worth more than the fix.
LLM Evaluation
Measuring whether a model or pipeline actually works, using error analysis, labeled datasets, and validated judges rather than a glance. The single most in-demand and least-taught AI engineering skill.
Error Analysis
Reading real failure traces and coding them into categories to find where a system actually breaks, before building any metric or dashboard. Most of eval effort should go here, not into tooling.
Golden Dataset
A curated set of inputs with known-correct expected outputs, used to measure quality and catch regressions when you change a prompt, a model, or the retrieval stack.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Agentic RAG
Agentic RAG is RAG where a language model runs the search in a loop. It decides whether to search, what to search for and where. Then it checks if the text it found answers the question.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
This page explains the idea. The "LLM-as-Judge and Its Failure Modes" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 210 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the LLM-as-Judge and Its Failure Modes lessoncourse 2
Take models from notebook to production, with labs on real models.