Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningAI Engineering · Advanced · LLM and GenAI Ops
Measuring whether a model or pipeline actually works, using error analysis, labeled datasets, and validated judges rather than a glance. The single most in-demand and least-taught AI engineering skill.
In short
LLM evaluation is measuring whether a language model feature gives good answers. You run a fixed set of labelled cases through it and score each output on a named property, such as 'faithful to the source', instead of reading a few replies and deciding it feels fine.
In short
LLM evaluation is measuring whether a language model feature gives good answers. You run a fixed set of labelled cases through it and score each output on a named property, such as 'faithful to the source', instead of reading a few replies and deciding it feels fine.
Learn it in the course
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers LLM Evaluation across a run of lessons, not one page. These 4 alone are about 239 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
Why evals, not observability, decide if your LLM feature is good, and why measuring quality is the core skill of AI engineering in 2026.
I wrote 90 new customer messages in six kinds the training data never showed: long emails, two requests in one message, other languages, decoy numbers, rare products and plain cancel requests. On the 53 that both annotators fully agreed on, the fine-tuned 0.5B got 22 whole tickets right, the fine-tuned 1.5B 21, the prompted 7B 24 and the prompted 14B 25. No fine-tune could be told apart from the 14B (smallest p 0.424). On all 15 cancel requests the two annotators disagreed about the category: a gap in the rules that no model can fix.
Four small models measured on one laptop with the same tests: gemma3:4b got 33 of 40 sums right, qwen2.5:3b 30, llama3.2:3b 23, and qwen3:4b 0, because it reasoned in its reply and ran out of its 32-token budget. Given 1,024 tokens it finished 22 sums and got all 22 right, at a median 926 tokens a sum. Only one pairwise gap was clearly bigger than luck. qwen2.5 wrote fastest, 38.2 tokens a second; gemma3 wrote 20.2.
The first lesson of the ML and AI lifecycle chapter. On 45,312 half-hours of electricity prices, I scored three rules that learn nothing against three trained models, on the future. Saying whatever the previous half-hour was scored 0.8484. The best model, boosted trees with the previous label as an input, scored 0.8193. The same trees on a shuffled test said 0.8830. Where the lazy rule fails, why the label is sticky by construction, and what balanced accuracy adds.
all part of
AI Engineering
210 lessons · about 219 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with Why Evals Are the Job: Moving From Vibes to Measurementpay once,
yours for life
An LLM feature is any part of a product where a large language model, a model that writes text one small piece at a time, produces the answer. Evaluation, usually shortened to evals, is how you find out whether those answers are good. Without it, quality is a feeling. You change the prompt, read a few replies, decide it seems better, and ship.
Many teams believe their logs already answer the question. Observability, meaning tools that record each request with its prompt, the retrieved text, the token count and the time taken, tells you what happened. It does not tell you whether it was right. A wrong answer and a right answer can leave exactly the same record: status 200, the same latency, the same token count, and no error.
This gap is common. The LangChain State of Agent Engineering survey asked more than 1,300 people who build agents what they had in place: 89 percent had observability and 52 percent had evals. Nearly everyone can watch their system, and only about half can grade it.
An eval has four parts. First, a fixed and versioned set of cases, each one an input plus a label that says what good looks like. Second, the system under test, meaning your prompt, model, retrieval and tools. Third, the output the system produced. Fourth, a scorer, which judges the output and returns a score with a reason.
The scorer is where evals differ from normal software tests. A unit test checks one exact answer, such as add(2, 2) must equal 4. LLM output is open-ended, so a hundred different summaries of one support ticket can all be correct, and the same question asked twice can come back in different words. There is no single correct string to compare against, so the scorer judges a property instead: is this summary faithful to its source, and does this answer address the question that was asked?
In practice, scorers come in three kinds. Simple rules check things like length, language or JSON shape. Reference checks compare against a known fact. An LLM-as-judge is a second model that scores the output against a written rubric, which scales to thousands of cases cheaply but is only worth trusting after you compare its grades with careful human labels on the same cases.
BLEU and ROUGE are older scores that count how many words or short phrases an answer shares with a reference answer. They assume there is one correct wording. For open-ended text that assumption breaks, and the result can be worse than useless.
Here is a worked example with a simple word-overlap score, counted by hand. The reference is 'The payment failed because the card had expired.' A correct paraphrase, 'Your card was out of date, so the charge did not go through', shares only two words with it, so it scores 2 x 2 / (13 + 7) = 0.20. A wrong answer, 'The payment expired because the card had failed to load', shares all seven reference words and scores 2 x 7 / (9 + 7) = 0.875. The wrong answer wins because a set of words has no order and cannot see that 'failed' and 'expired' swapped places.
In a small lab of five answer pairs, a score built on this idea disagreed with the human verdict on four of the five. A wrong instruction that reused the same eleven words in a different order got a perfect 1.00, and the worst correct answer got 0.18. Real BLEU and ROUGE look at runs of words and would give different exact numbers, but they still score the surface of the text, not its meaning.
A prompt change is never local. You tighten one instruction to fix the reply someone complained about, and the same edit moves every other case the model handles. The model gives no compiler error when your fix quietly breaks a case you were not looking at.
An eval is a leading signal: its score drops on the day of the change, before a user sees anything. Complaints, refunds and churn are lagging signals that move weeks later, after a worse product has been served to real people. The cost of having no evals is not zero. It is paid later, at the most expensive moment.
A suite that passes 100 percent is usually a warning, not good news. It often means the cases only cover the easy path, and none of the ways the system really breaks: a missed document, an ignored constraint, an invented number, or an answer to a nearby question. Small sets also mislead in the other direction. An 80 percent score on 100 examples really means somewhere between 72 and 88 percent, and in a simulation of 4,000 eval runs, a real 5-point improvement was detected only 15 percent of the time with 100 examples.
Build it in a fixed order, because each step needs the one before it. Start with error analysis, which means reading real failures. Turn those failures into a golden dataset of labelled cases. Then automate the scoring with a judge model, and last, check the judge against human labels. The biggest jump is the first one, from a feeling to a fixed set you can run again and compare.
Once the set exists, work in the style called eval-driven development. Write an eval from a real failure first, make one change, measure the whole set, and ship only if the number moves the right way. Change one thing at a time. If you change the prompt, the model and the retrieval together, you cannot say which one moved the score.
You do not need an eval when there is exactly one right output, such as a parser, a date format, a JSON schema or a calculation; a normal unit test is faster and exact. You need only a handful of cases you read yourself when you are exploring an idea nobody depends on yet. And no eval decides what a good answer means for your product. A person has to decide that and write it down first.
LangSmith
LangChain's hosted platform records traces of LLM applications and runs evaluations of an application over stored datasets of examples.
OpenAI Evals
An open-source framework published by OpenAI for defining and running evaluations of model outputs against sample sets.
promptfoo
An open-source command-line tool that runs a list of test cases with assertions against prompts and models and reports pass or fail for each case.
Ragas
An open-source Python library of evaluation metrics for retrieval-augmented generation, such as faithfulness and context precision.
Observability records what happened on each request: the prompt, the retrieved text, the tokens and the time. Evals judge whether the output was good. A wrong answer and a right answer can produce identical traces, so logging alone can never tell you the answer was correct.
Not for open-ended answers. They count words shared with one reference answer, so a correct answer in different words scores low and a wrong answer that reuses the reference's words can score high. In a small five-pair lab, a simple overlap score disagreed with a human on four of the five pairs.
LLM-as-judge means a second model grades each output against a written rubric. It is cheap and fast, which is why it is used at scale. Its scores are only worth acting on after you have compared them with careful human labels on the same cases and found that they agree.
Enough to detect the smallest change you care about. An 80 percent score on 100 examples has a 95 percent range of about 72 to 88 percent, and a simulation found a real 5-point gain was caught only 15 percent of the time at 100 examples. Catching it reliably took about 1,000.
Usually it means the suite is too easy. Real LLM features fail in specific ways, such as missing a document or inventing a number, and a suite that catches none of them is probably only testing the easy path. Ask which failure you are not testing for.
A Fine-Tune on New Cases: Messages Shaped Unlike Its Training, and a Gap No Model Can Fix
I wrote 90 new customer messages in six kinds the training data never showed: long emails, two requests in one message, other languages, decoy numbers, rare products and plain cancel requests. On the 53 that both annotators fully agreed on, the fine-tuned 0.5B got 22 whole tickets right, the fine-tuned 1.5B 21, the prompted 7B 24 and the prompted 14B 25. No fine-tune could be told apart from the 14B (smallest p 0.424). On all 15 cancel requests the two annotators disagreed about the category: a gap in the rules that no model can fix.
Comparing Models Fairly: Same Test, Different Results
Four small models measured on one laptop with the same tests: gemma3:4b got 33 of 40 sums right, qwen2.5:3b 30, llama3.2:3b 23, and qwen3:4b 0, because it reasoned in its reply and ran out of its 32-token budget. Given 1,024 tokens it finished 22 sums and got all 22 right, at a median 926 tokens a sum. Only one pairwise gap was clearly bigger than luck. qwen2.5 wrote fastest, 38.2 tokens a second; gemma3 wrote 20.2.
Start With the Dumbest Model: The Baselines a Model Must Beat, on the Future
The first lesson of the ML and AI lifecycle chapter. On 45,312 half-hours of electricity prices, I scored three rules that learn nothing against three trained models, on the future. Saying whatever the previous half-hour was scored 0.8484. The best model, boosted trees with the previous label as an input, scored 0.8193. The same trees on a shuffled test said 0.8830. Where the lazy rule fails, why the label is sticky by construction, and what balanced accuracy adds.
Small Rewordings: The Same Instruction, Seven Ways
Lesson 1's sorting instruction, reworded seven ways that mean the same thing, on 40 customer messages and two small models. qwen2.5:3b gave 32 to 39 usable answers of 40 and llama3.2:3b 4 to 31. Four orders of the category list gave 36 to 40 and 19 to 28. Only 27 of the 40 messages were usable under every wording on qwen, and 2 on llama. The same two wordings came top on both models, but on qwen only 2 of 21 pairs of wordings differed by more than luck.
Testing a Prompt on New Cases: Why the Score You Tuned On May Be Optimistic
Three prompts from this chapter, scored on the 40 messages the chapter was built around and on 40 new messages written before any prompt was scored on them, on two small models. No prompt scored higher on the new messages. qwen2.5:3b went from 36, 40 and 39 usable answers to 34, 40 and 36; llama3.2:3b from 20, 36 and 33 to 17, 32 and 31. The best prompt held on qwen and lost 4 on llama. Of the 50 wrong answers on the new messages, 26 were account messages, none of which used the word account.
LLM-as-Judge
Using a language model to score another model's output. Fast and scalable, but it has real failure modes such as self-preference and position bias, so the judge itself must be validated against human labels.
Error Analysis
Reading real failure traces and coding them into categories to find where a system actually breaks, before building any metric or dashboard. Most of eval effort should go here, not into tooling.
Golden Dataset
A curated set of inputs with known-correct expected outputs, used to measure quality and catch regressions when you change a prompt, a model, or the retrieval stack.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Agentic RAG
Agentic RAG is RAG where a language model runs the search in a loop. It decides whether to search, what to search for and where. Then it checks if the text it found answers the question.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
This page explains the idea. The "Why Evals Are the Job: Moving From Vibes to Measurement" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 210 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Why Evals Are the Job: Moving From Vibes to Measurement lessoncourse 2
Take models from notebook to production, with labs on real models.
Asked about LLM Evaluation in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (180 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: PayPal. Download at once from My packs.
