May we use cookies so Meta, Google, LinkedIn and ChatGPT see your visit and we learn which ads work? PrivacyMay we use cookies to see which pages help, and let Meta, Google, LinkedIn and ChatGPT see your visit, so we know which of our ads work? That way we spend less on ads that do not work. We do not sell your data, and the site works the same if you decline. You can change your answer at any time under Cookie settings at the foot of the page. Privacy policy
Loading...
AI Engineering · Ml Lifecycle
Free preview: the first part of this 23-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by card or PayPal
See the Generative AI and Agentic AI courseread the rest of this lesson in
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by card or PayPal
See the Generative AI and Agentic AI coursein Ml Lifecycle
This lesson is part of the AI Engineering course. Unlock the full course, a separate one-time payment, for lifetime access.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime AccessFoundation lessons of this course are free, no payment needed. Try them now
Imagine a running club that wants to sign one new runner for the autumn season. Two hundred people come to a one-day tryout in spring. Each runs one race, and the club signs whoever is fastest. It sounds fair. But that day there was a strong wind, and it blew harder at some moments than at others. The fastest runner may simply have had the wind at her back. The more people who run, the more likely it is that the fastest time is partly wind.
There is a second problem, and it is quieter. The tryout was on a short, flat track in spring. The season is in autumn, on long hilly courses. A runner who is great on a flat sprint may be ordinary on a hill. So the club can be wrong twice: once because the winning time was partly luck, and again because the tryout tested a different kind of race from the one that matters.

This lesson does the same thing to a program that learns from examples. I tried 200 versions of one program and kept the one that looked best. Then I checked it on later months that had played no part in the choice. The winner looked better than a random pick on the months I chose with, and worse on the months that came after. The rest of the lesson measures how much of that was the wind, and how much was the wrong track.