May we use cookies so Meta, Google, LinkedIn and ChatGPT see your visit and we learn which ads work? PrivacyMay we use cookies to see which pages help, and let Meta, Google, LinkedIn and ChatGPT see your visit, so we know which of our ads work? That way we spend less on ads that do not work. We do not sell your data, and the site works the same if you decline. You can change your answer at any time under Cookie settings at the foot of the page. Privacy policy
Loading...
AI Engineering · Why Production Breaks
Free preview: the first part of this 23-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by card or PayPal
See the Generative AI and Agentic AI courseread the rest of this lesson in
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by card or PayPal
See the Generative AI and Agentic AI coursein Why Production Breaks
This lesson is part of the AI Engineering course. Unlock the full course, a separate one-time payment, for lifetime access.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime AccessFoundation lessons of this course are free, no payment needed. Try them now
Imagine you drive an old car whose fuel gauge has a loose wire. Nothing warns you. The needle still points somewhere, the engine still runs, and every time you glance down the dashboard shows you a number. One day the needle has sat at half a tank for a week of long drives. Another day it points far past full, to a mark that is not even printed on the dial. The car does not stop you. It keeps driving until the tank is really empty, and then it stops in the middle of the road.
Looking back, there were signs. A needle that does not move for a week of driving is odd. A needle past the end of the dial is impossible. You did not need to know how much fuel was really in the tank to notice either of those. You only needed to know what a normal needle looks like.

A model that has learned from examples is like that gauge. It always shows a number. When one of its inputs breaks, it does not stop and it does not warn you. This lesson asks the question the driver should have asked: which simple checks, run every day on what the model is given and what it answers, would have noticed the break before the real answers came in? I tested six such checks against seven kinds of break. Four of them did well. One of them raised an alarm every single day, broken or not. And the worst break got past all six.