AI Engineering · Agents In Production
Free preview: the first part of this 16-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
AI Engineering
Unlock this lesson and the 199 other paid lessons
$49 once, by PayPal
See the AI Engineering courseread the rest of this lesson in
AI Engineering
Unlock this lesson and the 199 other paid lessons
$49 once, by PayPal
See the AI Engineering coursein Agents In Production
This lesson is part of the AI Engineering course. Unlock the full course, a separate one-time payment, for lifetime access.
This idea carries a full system design question on its own. Each walks through the full answer.
One-time payment: UPI, card or netbanking
Get Lifetime AccessFoundation lessons of this course are free, no payment needed. Try them now
An agent that can issue a refund is one confident misclassification away from issuing one it should not. The model reads a message that says "this is outrageous, I want my money back for order 1001," decides on its own that this is a refund, and calls the refund tool. Most of the time it is right. The time it is wrong, because the order was already refunded, or the message was quoting someone else, or the customer is not who they claim to be, the money is gone and you find out from finance.
The reflex is to make the model better at that decision: a sharper prompt, a bigger model, an example of when not to refund. The reflex is wrong, and the earlier lessons in this chapter say why. The lesson showed that a good model still emits bad calls on ordinary inputs. The first lesson showed that per-step reliability caps out below 1.0, so a decision the model gets wrong some fraction of the time will be wrong some fraction of the time no matter how you prompt it.
Senior agent systems do something else. They take the decision away from the model. The model still reads the message and still says what it thinks it is: a refund request. But it does not decide whether the refund happens. Code decides that, by looking up the order, checking it is refundable, and confirming the customer owns it. The model explains. Code arbitrates.

That word, arbitrate, is the whole lesson. To arbitrate is to make the binding decision that changes the world: money moves, a row deletes, an email sends, the workflow takes one branch instead of another. Everything here is a rule for keeping that decision in code and leaving the model with the part only language can do.