RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
What is RAG (Retrieval-Augmented Generation)?
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
RAG (Retrieval-Augmented Generation) is a advanced concept that sits in the LLM and GenAI Ops area of system design. Engineers reach for it whenever they need to reason about real-world trade-offs in that space, not just for textbook correctness, but because real production systems at companies like Netflix, Amazon, and Google make these decisions every day.
If you want to go deeper than this definition, with diagrams, code, and a quiz to lock it in, work through the "Production RAG and LLM Systems" lesson linked below. It walks through the why, the mechanism, the trade-offs, and how the giants actually use it in production.
Learn RAG (Retrieval-Augmented Generation) in depth
Full interactive lesson with diagrams, code examples, real-world references, and a quiz.
Open the Production RAG and LLM Systems lessonRelated lessons
Lessons that touch on RAG (Retrieval-Augmented Generation) as part of a larger topic.
Design a RAG System at Scale
Design a production Retrieval-Augmented Generation system - vector databases, chunking strategies, retrieval pipelines, reranking, hybrid search, caching, evaluation, and cost management
capstone · capstone
Fine-Tuning vs RAG vs Prompting: Choosing Your Approach
Fine-tuning, RAG, and prompting each adapt an LLM differently. Here is what each one changes and a framework for picking the right one.
ml-advanced · llm genai ops
The RAG Scale Cliff: What Breaks Between 100 and 5 Million Documents
Why RAG fails in production as the corpus grows: a system that shines on a 100-document demo gets subtly worse at 5 million, offline metrics stay silent, and here are the levers that fight the RAG scale cliff.
ml-intermediate · retrieval rag
The RAG Retrieval Cliff: Engineering Recall Back at Scale
The corpus-growth cliff is one equation, lambda equals N times p. Learn the fixes that hold recall at ten million chunks: filtering, partitioning, coarse-to-fine and hierarchical retrieval, ANN tuning, and what each costs.
ml-intermediate · retrieval rag
RAG Evaluation: Localizing Failure Between Retrieval and Generation
Recall@k, context precision, and faithfulness split every wrong RAG answer into a retrieval bug or a generation bug, each with a different fix.
ml-intermediate · retrieval rag
See also
Related glossary terms you might want to look up next.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
Vector Database
A database optimized for storing and querying high-dimensional vectors. Powers AI applications like semantic search and recommendation systems.
Reranking
A second retrieval stage where a cross-encoder scores each candidate chunk against the query directly. Slower than the first-stage retriever but far more precise, so it runs only over the top few dozen results.
Hallucination
When a language model produces fluent, confident text that is not supported by any source and is often simply false. Retrieval, grounding, and evaluation are the main defenses.
ANN (Approximate Nearest Neighbor)
Search that finds vectors close to a query without scanning every item, trading a small chance of missing a true neighbor for a large gain in speed. HNSW and IVF are the common indexes.
HNSW
Hierarchical Navigable Small World, a graph-based nearest neighbor index that gives high recall at low latency by navigating layers of links, at the cost of more memory than IVF.