May we use cookies so Meta, Google, LinkedIn and ChatGPT see your visit and we learn which ads work? PrivacyMay we use cookies to see which pages help, and let Meta, Google, LinkedIn and ChatGPT see your visit, so we know which of our ads work? That way we spend less on ads that do not work. We do not sell your data, and the site works the same if you decline. You can change your answer at any time under Cookie settings at the foot of the page. Privacy policy
Loading...
AI Engineering · Why Production Breaks
Free preview: the first part of this 25-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by card or PayPal
See the Generative AI and Agentic AI courseread the rest of this lesson in
Generative AI and Agentic AI Engineering
Unlock this lesson and the 208 other paid lessons
$35 once, by card or PayPal
See the Generative AI and Agentic AI coursein Why Production Breaks
This lesson is part of the AI Engineering course. Unlock the full course, a separate one-time payment, for lifetime access.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime AccessFoundation lessons of this course are free, no payment needed. Try them now
Imagine a small bakery that opened in a quiet part of town. In its first year the owner kept a careful notebook: for every hour of every day, how many loaves were sold. By the end of the year she knows her shop well. Monday mornings are slow. Saturday mid-morning is the rush. Rainy days are quiet. She uses the notebook to decide how much dough to make each morning, and it works.
Then, in her second year, a new office building opens across the road. Hundreds of new people walk past every morning. The notebook still says what a Monday in March looked like last year, and the owner still bakes for last year's Monday in March. By half past eight the shelves are empty. The notebook was not wrong about last year. It was only about last year, and she was using it for this one.

Nothing in the notebook tells her that the world outside has changed. She finds out from the empty shelves. The fix is simple to say: keep writing in the notebook, and plan from the newest pages, not only the oldest ones. It is harder to do well. How often should she look again? Should she throw away last year's pages, or keep them? And what about the very first morning after the office opened, when there were no new pages yet?
This lesson asks those questions of a computer program that guesses how many bikes a city rents in an hour. I taught it one year, then asked it about every hour of the next year, and I tried three ways of keeping it up to date. The program that was never updated missed by about twice as much as the one updated every month, and most of its error came from guessing too low: it was under the real count in 86% of the hours.