Tool Calling
When a model emits a structured request to run a function, and an orchestrator executes it and feeds the result back. The model never runs anything itself, so the orchestrator validates every call.
What is Tool Calling?
When a model emits a structured request to run a function, and an orchestrator executes it and feeds the result back. The model never runs anything itself, so the orchestrator validates every call.
Tool Calling is a advanced concept that sits in the LLM and GenAI Ops area of system design. Engineers reach for it whenever they need to reason about real-world trade-offs in that space, not just for textbook correctness, but because real production systems at companies like Netflix, Amazon, and Google make these decisions every day.
If you want to go deeper than this definition, with diagrams, code, and a quiz to lock it in, work through the "LLM Agents in Production" lesson linked below. It walks through the why, the mechanism, the trade-offs, and how the giants actually use it in production.
Learn Tool Calling in depth
Full interactive lesson with diagrams, code examples, real-world references, and a quiz.
Open the LLM Agents in Production lessonRelated lessons
Lessons that touch on Tool Calling as part of a larger topic.
Tool Calling Is a Contract, and the Model Will Break It
The model never runs your functions. It emits text proposing a call, and sometimes that text is wrong. How to validate every tool call against a schema before it runs, and reject cleanly so the model self-corrects.
ml-intermediate · agents in production
Design an AI Agent Platform
Design a production AI agent orchestration platform - MCP, A2A protocol, tool calling, memory systems, multi-agent workflows, cost management, and safety guardrails
capstone · capstone
Evaluating RAG and Agents: One Score Detects, Components Localize
One end-to-end score tells you something broke, not what. Split RAG and agent quality into retrieval, generation, tool, and trajectory evals that localize failures and score the path, not just the answer.
ml-advanced · evals
MCP in Production: What the Protocol Buys and Costs
MCP turns N times M tool integrations into N plus M, but every attached server rents context-window tokens on every request. When it pays, when it does not.
ml-intermediate · agents in production
See also
Related glossary terms you might want to look up next.
AI Agent
A language model placed in a loop: it decides an action, calls a tool, reads the result, and repeats until done. The hard part is that per-step errors compound over many steps.
MCP (Model Context Protocol)
A standard for exposing tools and data to a model in a uniform, discoverable way, so integrations are not hand-rolled per model. Widely adopted in 2026, with its own security concerns.
Structured Output
Constraining a model to return valid JSON matching a schema, so downstream code can rely on it. The foundation of reliable tool calling.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
ANN (Approximate Nearest Neighbor)
Search that finds vectors close to a query without scanning every item, trading a small chance of missing a true neighbor for a large gain in speed. HNSW and IVF are the common indexes.