Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningAI Engineering · Advanced · LLM and GenAI Ops
An AI system stops at chosen points and waits for a person to approve, change or reject its next action before it happens. Used for actions that cost money or cannot be undone.
In short
Human in the loop (HITL) means an AI system stops at chosen points. It waits for a person to approve, change or reject its next action before it does it. In AI agents it is used for actions that are costly or cannot be undone, such as paying a refund, sending an email to a customer or deleting data.
In short
Human in the loop (HITL) means an AI system stops at chosen points. It waits for a person to approve, change or reject its next action before it does it. In AI agents it is used for actions that are costly or cannot be undone, such as paying a refund, sending an email to a customer or deleting data.
Learn it in the course
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers Human in the Loop across a run of lessons, not one page. These 4 alone are about 230 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
The operator's view of running LLM agents: deployment topology, budgets and governors, tracing, canary rollout, human escalation, and the golden signals you page on.
An AI agent is a language model in a loop, and its reliability compounds: 95 percent per step over 20 steps is only 36 percent. Why agents fail on long tasks, and the levers that fight it.
A prompt template filled four ways, on lesson 1's 40 messages and 7 tricky ones, with no model. str.format crashed on all 47 inputs, because the template held a JSON example with braces. A two-step format crashed on 2 inputs a customer typed and, on a third, copied an internal note into the customer's message. replace and an f-string had no problems. Then five versions of lesson 1's prompt that differ by a space, a new line, a label or a pasted copy: qwen2.5:3b gave 39 to 40 usable answers of 40, llama3.2:3b 33 to 39, and no change beat luck.
Streaming feature pipelines use a watermark to decide how long to wait for late events. The shop data has no arrival times, so I simulated them: one delay per invoice from two stated delay models, 20 seeds. I tried watermarks from 0 to 48 hours and two policies for late events. When late events were kept for later results, at most about 275 of 26,851 test rows had a wrong feature, and the change in average precision could not be told apart from zero at any watermark. When late events were dropped for good, heavy simulated delays changed 8,717 rows and cost 0.0038; waiting 48 hours cut that to 0.0013.
all part of
AI Engineering
210 lessons · about 219 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with LLM Agents in Production: The Model in a Looppay once,
yours for life
A loop is a set of steps that repeats. An AI agent works in a loop: it decides an action, does it, looks at the result, and decides again. An AI agent is a program that lets a large language model (LLM), the kind of model behind ChatGPT, choose its next step.
Human in the loop means a person is one of the steps. At chosen points the agent stops. It shows a person what it plans to do. The person says yes, changes it, or says no. Only then does the agent carry on.
Why not let the agent do everything? Because a model can be wrong in a confident voice. Reading a wrong answer costs little. Paying a wrong refund, emailing the wrong customer or deleting the wrong rows costs real money and trust. A person in the loop is a check placed exactly where a mistake would be expensive.
The same idea is older than AI agents. People label data (mark each example with the right answer) so models can learn from it. A person also checks a loan that a model marked as risky. In agents, the meaning is narrower: a run pauses before an action until a person decides.
People search for "human in the loop vs human on the loop" because the two sound alike. They put the person in different places.
The law points the same way for risky AI. Article 14 of the EU AI Act, the European Union's law on AI, says high-risk AI systems must be built so that people can "effectively" oversee them while they are in use. It says the people overseeing the system must be able to "disregard, override or reverse" its output, meaning ignore it, replace it or undo it, and to stop it with a "stop" button or a similar procedure. It also warns about automation bias: the habit of trusting a machine's answer too much.
Four things have to be true for an agent to wait for a person properly.
Anthropic's 2024 guide "Building effective agents" describes the same pattern. It says agents "can then pause for human feedback at checkpoints or when encountering blockers." It also advises a stopping condition, such as a maximum number of steps, so a run never loops forever.
LangGraph builds this in. A step calls interrupt(). The run stops, its state is saved by a checkpointer (the part that saves state) under a thread id (the name the saved run is filed under), and the call returns a special __interrupt__ value to your app. Your app shows it to a person. When the person answers, you call the graph again with Command(resume=...) and the same thread id. The answer becomes the return value of interrupt(), and the run carries on.
This example pays refunds of Rs 1,000 or less at once, and asks a person about anything bigger. It has no LLM in it, so it runs the same way every time. It ran on langgraph 1.2.16.
refund_approval.py
from typing import TypedDict
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import END, START, StateGraph
from langgraph.types import Command, interrupt
PAID: list[str] = []
class Refund(TypedDict, total=False):
order_id: str
amount: int
decision: str
def ask_human(state: Refund) -> dict:
if state["amount"] <= 1000: # small refunds need no person
return {"decision": "auto-approved"}
decision = interrupt({"question": "Approve this refund?", "order": state["order_id"], "amount": state["amount"]})
return {"decision": decision} # whatever the person sent back
def pay(state: Refund) -> dict:
if state["decision"] in ("approve", "auto-approved"):
PAID.append(state["order_id"]) # the money moves only after the decision
return {}
hb = StateGraph(Refund)
hb.add_node("ask_human", ask_human)
hb.add_node("pay", pay)
hb.add_edge(START, "ask_human")
hb.add_edge("ask_human", "pay")
hb.add_edge("pay", END)
app = hb.compile(checkpointer=InMemorySaver())
cfg = {"configurable": {"thread_id": "refund-7"}}
first = app.invoke({"order_id": "A-7", "amount": 4500}, cfg)
print("paused, asking:", first["__interrupt__"][0].value)
print("paid so far:", PAID)
app.invoke(Command(resume="approve"), cfg) # hours later, from a different process if needed
print("paid after approval:", PAID)Output, captured 2026-10-18
paused, asking: {'question': 'Approve this refund?', 'order': 'A-7', 'amount': 4500}
paid so far: []
paid after approval: ['A-7']
The first call stopped at the interrupt. Nothing was paid. The second call resumed the same thread with "approve", and only then did order A-7 appear in the paid list. In a real app the two calls can be hours apart and on different servers, as long as the checkpointer is a database such as PostgreSQL. InMemorySaver, used here, loses everything when the program stops.
If you use LangChain's ready-made agent, its HumanInTheLoopMiddleware does the same job per tool. Middleware is code that sits between the agent and its tools and can stop a call. You list which tools need a person and which decisions are allowed: approve, edit, reject or respond. Its docs say you must set a checkpointer for it to work.
This one surprises almost everyone. When a paused step resumes, LangGraph does not continue from the exact line where it stopped. The docs say "the runtime restarts the entire node from the beginning." So any code above interrupt() runs again.
The same node, counted
CALLS = {"before": 0, "after": 0}
def node(state: T) -> dict:
CALLS["before"] += 1 # runs on the first pass AND again on resume
answer = interrupt("Send the email?")
CALLS["after"] += 1 # runs once, after the answer
return {"ok": answer}
# ...build and compile a one-node graph with a checkpointer, then:
tg.invoke({}, c)
tg.invoke(Command(resume="yes"), c)
print("code before interrupt ran:", CALLS["before"], "times | after:", CALLS["after"], "time")Output, captured 2026-10-18
code before interrupt ran: 2 times | after: 1 timeThe line above interrupt() ran twice. If that line had sent an email or charged a card, the customer would have got two. The docs say side effects (actions outside the program, such as sending an email) before an interrupt "should (ideally) be idempotent". Idempotent means doing it twice has the same effect as doing it once. The safe habit is simpler: put the action after the decision, or in its own step, as the refund example does with its separate pay step.
Two more rules from the docs. If one step calls interrupt() more than once, the answers are matched to the calls by their order, so keep that order fixed. And the run needs both a checkpointer and a thread id, or there is nothing to resume.
Use human in the loop when one wrong action is expensive, when a law or a contract says a person must decide, and while a new agent is still earning trust. Many teams start with every risky action approved, then measure how often people change or reject them. Where people almost never disagree, they raise the limit, step by step.
Do not use it for read-only steps, for answers a person will read anyway, or where speed matters more than a rare mistake. There, watching from on the loop is enough.
For a full design that puts this in a real product, read the customer support agent interview guide and the AI agent interview guide.
LangGraph
interrupt() pauses a step and saves the state; Command(resume=...) carries on with the person's answer. Needs a checkpointer and a thread id.
LangChain
HumanInTheLoopMiddleware lets you mark which tool calls need a person, with the decisions approve, edit, reject and respond.
Anthropic
Its 2024 guide Building effective agents says agents can pause for human feedback at checkpoints, and advises stopping conditions such as a maximum number of steps.
EU AI Act, Article 14
Requires high-risk AI systems to be built so people can oversee them, override or reverse their output, and stop them.
Slack
A common place to send the approval request, with approve and reject buttons that call back into the paused run.
Human in the loop means an AI system stops at chosen points and waits for a person to approve, change or reject its next action before it happens. It is used where a wrong action is expensive or cannot be undone, such as payments, refunds, emails to customers and deleting data.
In the loop, the system waits for a person before it acts, so nothing happens without a yes. On the loop, the system acts by itself while a person watches and can stop it or undo the result. In the loop suits costly, one-way actions; on the loop suits many cheap, reversible ones.
A step calls interrupt(). The run stops, a checkpointer saves its state under a thread id, and your app gets the question under __interrupt__. When a person answers, you call the graph again with Command(resume=answer) and the same thread id, and the run carries on from that step.
Because on resume LangGraph runs the whole step again from the top, and the docs say so. Any code above interrupt() runs once before the pause and once after. Put side effects such as payments or emails after the decision, or in a separate step.
Actions that move money, change an account, contact a customer, delete data, or cannot be undone. Read-only steps such as searching or looking up an order should not wait. Decide this in code, per tool and per amount, not by asking the model.
AI Agent
A language model placed in a loop: it decides an action, calls a tool, reads the result, and repeats until done. The hard part is that per-step errors compound over many steps.
LangGraph
An open-source framework for building AI agents as a graph of steps that share one state. Nodes do the work, edges pick the next step, and a checkpointer saves the state so a run can pause, resume and remember.
Guardrails
The checks around a model that screen its input for abuse and its output for grounding and safety, plus limits on what its tools can do. Containment, not just filtering, is what actually holds.
Prompt Injection
An attack where untrusted text, often returned by a tool, carries an instruction the model obeys, because the model cannot reliably tell instructions from data. The top security risk for agents, and unsolved by prompting alone.
Tool Calling
When a model emits a structured request to run a function, and an orchestrator executes it and feeds the result back. The model never runs anything itself, so the orchestrator validates every call.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
This page explains the idea. The "LLM Agents in Production: The Model in a Loop" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 210 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the LLM Agents in Production: The Model in a Loop lessoncourse 2
Take models from notebook to production, with labs on real models.
Asked about Human in the Loop (HITL) in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (173 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
