Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningAI Engineering · Advanced · LLM and GenAI Ops
A curated set of inputs with known-correct expected outputs, used to measure quality and catch regressions when you change a prompt, a model, or the retrieval stack.
In short
A golden dataset is a fixed, hand-chosen set of inputs, each paired with an answer a person stands behind. You run it before and after every prompt, model or retrieval change and compare the results case by case, so a change that quietly breaks something is caught before it ships.
In short
A golden dataset is a fixed, hand-chosen set of inputs, each paired with an answer a person stands behind. You run it before and after every prompt, model or retrieval change and compare the results case by case, so a change that quietly breaks something is caught before it ships.
Learn it in the course
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers Golden Dataset across a run of lessons, not one page. These 4 alone are about 195 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
A golden dataset is a curated set of inputs and known-good answers you run before and after every change, so a prompt tweak that quietly breaks things cannot ship.
A metric that never fails cannot detect anything, and a case won by 0.001 is recorded as the same pass as a case won by 0.05. Measure the difficulty of your eval set before you trust its pass rate.
Generating more eval cases grows the set along the dimensions you described. On a real set, 80% of the difficulty came from near-duplicate documents in the corpus, which no question generator can reach.
An eval set is described by how many cases it has. Join it against your index instead: on a real set, 5 of 15 documents were never the right answer to anything, and never appeared in any trace at all.
all part of
AI Engineering
210 lessons · about 219 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with Building a Golden Dataset: The Set That Catches Regressionspay once,
yours for life
A golden dataset is the answer key for an AI system. It is a curated list of inputs, and each input comes with an expected output that you already know is correct. You run the system on every case and check whether its answers still match. Other names for the same thing are golden set, eval set and regression set.
The important word is curated. A dump of production logs is not a golden set, because those rows were chosen by chance and none of them has an expected answer, so nothing can be graded. In a golden set every case is there for a reason, and every case carries an expected output. That output can be a label such as 'spam', a fact the answer must contain, a policy it must not break, or a full reference answer.
Each case holds three things. The input exactly as the system sees it, with the real typo or mixed language left in. The expected output a person stands behind. And the failure mode it belongs to, so the results can be read by category instead of as one number.
The cases come from error analysis, the step where you read real traces and named the ways your system fails. Pull cases for each named failure mode, add a block of ordinary everyday requests, and add a small block of adversarial inputs, meaning inputs designed to trick the system. The everyday block is not padding. It catches a change that fixes an edge case but breaks the normal request.
Do not fill the set with a random sample of traffic. Most traffic is the easy path, so a failure that hits 1 percent of requests shows up about twice in 200 random rows. If a change breaks it completely, the overall score moves by about one point and disappears into normal run-to-run noise. A stratified set fixes the count on purpose, for example 20 cases per failure mode, so a break in any mode moves its own bucket sharply.
Size follows coverage. A few dozen to a few hundred well-chosen cases usually beat thousands of random ones, because every expected answer needs a person to stand behind it, and 150 answers you trust are sharper than 3,000 nobody checked. Size still limits what you can detect. In a simulation of 4,000 eval runs from an 80 percent baseline, a real 5-point gain was caught 15 percent of the time with 100 cases and about 83 percent of the time with 1,000, while a 2-point gain never reached 80 percent detection even at 5,000.
An expected answer written by one person carries that person's reading of the policy, including where they are quietly wrong. So send each candidate case to several reviewers and take the majority. A clear majority becomes the frozen expected answer.
A close split, such as three votes against two, means the case sits on a line your definition does not settle. Do not average it away. Escalate it to one decider, and write the reason into the labelling guideline with a worked example, so the next similar case is labelled the same way. To check the reviewers themselves, mix in a few 'salted gold' cases whose answer you already know; a reviewer who misses them is not reading carefully.
Then freeze the set and give it a version number, and store it next to the code it protects. Every run should name the exact version it graded. If the set changes silently between two runs, you are comparing two different exams. Treat an edit to an expected answer like an edit to a test assertion, and review it the same way.
Take eight refund questions with known answers. The old prompt passes 6 of 8. The new prompt also passes 6 of 8, so a dashboard that tracks one number calls the change safe. Compare case by case and the picture changes: the new prompt fixed two cases, a defective item and a recalled product, and broke two others, a gift card and a final-sale item. Those two breaks are real regressions, and the average hid them completely.
This is why a golden set is run as a gate in continuous integration, the automatic checks that run on every pull request. The build runs the whole set, compares each case with the last known-good run, and blocks the merge if any protected case now fails, naming the cases that broke.
A public example shows why coverage matters as much as the gate. Anthropic has published a postmortem about an April 2026 change to the Claude Code system prompt that asked for 25 words or fewer between tool calls. Their standard evaluations showed no regression, so it shipped. After user complaints, a broader set of evaluations showed a 3 percent drop for two model versions, and the change was reverted. The case that broke was not in the set that ran.
A golden set only covers failures you have already found, so it goes out of date from the day you finish it. This is called rot. The fix is a habit: when production shows a failure the set did not expect, capture that trace, give it a correct answer, file it under a failure mode, and add it as the next version. Loosen labels only by reading. In one measured case, a strict retrieval label failed 9 of 30 real traces, a rule that accepted any document named like the labelled one turned 70 percent into 100 percent, and reading the nine rows showed every one was a real miss.
Never let golden cases into training data or into the few-shot examples pasted into a prompt. If the model has seen the answer, it passes by memory and the score measures nothing. Both leaks feel like good engineering at the time, so keep the set in its own store and exclude it from every training pipeline by design.
When a failure mode is real but too rare to fill with real traces, you can write or generate synthetic inputs for it. A person must check every synthetic case: that the input really represents the failure, and that the expected answer is right on its merits. If a model writes both the input and the answer, it builds its own blind spots into the key. And when your goal is to estimate overall production quality rather than catch regressions, a random sample of traffic is the right tool; you need both, and they are different sets.
promptfoo
An open-source tool that runs a file of test cases with expected values or assertions against prompts and models, and is commonly run in CI.
LangSmith datasets
LangSmith stores datasets of example inputs with reference outputs and runs experiments that score an application against them.
OpenAI Evals
OpenAI's open-source eval framework reads sample files in which each input can carry an ideal answer to compare against.
DVC
Data Version Control is an open-source tool that versions data files alongside a git repository, which suits keeping a frozen, tagged eval set next to the code.
It is a curated set of inputs, each with a known-correct expected output, that you run before and after every change. It works like a regression test suite. The difference from normal software tests is that the comparison is sometimes a rule or a judge model instead of an exact string match.
Big enough to cover every failure mode you know about plus the everyday path, with answers you trust, and no bigger. A few dozen to a few hundred cases is common. If you need to detect small improvements, check the numbers: in one simulation a 5-point gain needed about 1,000 cases to be caught reliably.
Because traffic is mostly easy requests. A failure that affects 1 percent of requests gets only a couple of cases in a random sample, so breaking it barely moves the score. A stratified set gives each failure mode enough cases that a break is obvious.
Run the same frozen version of the golden set before and after the change, then compare the results case by case. An unchanged average can hide equal numbers of fixes and breaks. In an eight-case example, both versions passed 6 of 8 while two cases had broken.
Yes, for failure modes too rare to fill with real traces, as long as a person checks every synthetic case. If a model writes both the test inputs and the expected answers, it tends to encode its own mistakes into the answer key.
Case Difficulty: Why a Set Can Pass Everything and Tell You Nothing
A metric that never fails cannot detect anything, and a case won by 0.001 is recorded as the same pass as a case won by 0.05. Measure the difficulty of your eval set before you trust its pass rate.
Synthetic Cases: A Generator Writes Questions, Not Your Index
Generating more eval cases grows the set along the dimensions you described. On a real set, 80% of the difficulty came from near-duplicate documents in the corpus, which no question generator can reach.
Coverage: The Third of Your Index That Nothing Tests
An eval set is described by how many cases it has. Join it against your index instead: on a real set, 5 of 15 documents were never the right answer to anything, and never appeared in any trace at all.
A Repaired Golden Set Reported 100%. The System Earned 70%.
A strict golden label failed 9 of 30 real traces. A rule that accepts every document named after the label turned 70% into 100%. Reading the nine rows showed every one was a real miss. Loosen labels by reading, never by a rule about names.
LLM Evaluation
Measuring whether a model or pipeline actually works, using error analysis, labeled datasets, and validated judges rather than a glance. The single most in-demand and least-taught AI engineering skill.
Error Analysis
Reading real failure traces and coding them into categories to find where a system actually breaks, before building any metric or dashboard. Most of eval effort should go here, not into tooling.
LLM-as-Judge
Using a language model to score another model's output. Fast and scalable, but it has real failure modes such as self-preference and position bias, so the judge itself must be validated against human labels.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Agentic RAG
Agentic RAG is RAG where a language model runs the search in a loop. It decides whether to search, what to search for and where. Then it checks if the text it found answers the question.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
This page explains the idea. The "Building a Golden Dataset: The Set That Catches Regressions" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 210 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Building a Golden Dataset: The Set That Catches Regressions lessoncourse 2
Take models from notebook to production, with labs on real models.