May we use cookies so Meta, Google, LinkedIn and ChatGPT see your visit and we learn which ads work? PrivacyMay we use cookies to see which pages help, and let Meta, Google, LinkedIn and ChatGPT see your visit, so we know which of our ads work? That way we spend less on ads that do not work. We do not sell your data, and the site works the same if you decline. You can change your answer at any time under Cookie settings at the foot of the page. Privacy policy
Loading...
This idea carries a full system design question on its own. Each walks through the full answer.
AI Engineering · Ml Lifecycle
Free preview: the first part of this 26-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by card or PayPal
See the Generative AI and Agentic AI courseread the rest of this lesson in
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by card or PayPal
See the Generative AI and Agentic AI coursein Ml Lifecycle
This lesson is part of the AI Engineering course. Unlock the full course, a separate one-time payment, for lifetime access.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime AccessFoundation lessons of this course are free, no payment needed. Try them now
Imagine a teacher who gives his class a short test every Friday. On Monday he gets the marks back, sees which topics the class got wrong, and teaches those again that week. It works well. The class drifts off course for a few days at most before he notices and steers it back.
Now imagine the tests have to be sent away to be marked, and the marks come back three months later. Every week he still teaches, and every week the class still takes a test. But the marks he reads this Monday are from a test the class took in the spring. He is correcting mistakes his students made long ago, some of which they have already grown out of, and he has no idea at all how last Friday went.

Two things got worse for him at once. The first is learning: his lesson plans are built on old evidence, so they fit the class of three months ago, not the class in front of him. The second is knowing: if the class is struggling right now, he will not find out for three months. Both are the same problem seen from two sides. The answers that tell him how he is doing arrive late.
This lesson is about the same problem in a program that learns from examples. I kept one model, retrained it every week exactly as in lesson 6, and made its right answers arrive later and later. Then I measured what it cost the model, and what it cost the people watching it.