Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningAI Engineering · Foundation · LLM and GenAI Ops
A second retrieval stage where a cross-encoder scores each candidate chunk against the query directly. Slower than the first-stage retriever but far more precise, so it runs only over the top few dozen results.
In short
Reranking is a second search stage: a fast retriever returns many candidates, then a slower, more accurate model scores each one against the question and keeps the best few. It fixes answers that were found but ranked too low, not answers never found.
In short
Reranking is a second search stage: a fast retriever returns many candidates, then a slower, more accurate model scores each one against the question and keeps the best few. It fixes answers that were found but ranked too low, not answers never found.
Learn it in the course
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers Reranking across a run of lessons, not one page. These 3 alone are about 200 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
Retrieve 50, rerank with a cross-encoder, keep 5. Why a second-stage reranker is the cheapest, highest-impact fix for a RAG system that keeps missing the right chunk.
RAG triples a chat bill through retrieved context. Cut it with a semantic cache, embedding reuse, and a reranker, without serving confidently wrong answers.
Retrieval quality cannot exceed the quality of the text you extracted. Ingestion is the true ceiling of a RAG system, upstream of chunking, reranking, and hybrid search.
all part of
AI Engineering
210 lessons · about 219 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with Reranking: The Highest-Return Change in Your RAG Stackpay once,
yours for life
A common failure in retrieval-augmented generation (RAG), where a model answers from documents found by a search step, looks like this: the right text was found, but it was ranked too low to reach the model. A support bot that passes only the top five chunks to the model never sees an answer sitting at position twelve.
This is a ranking problem, not a recall problem, and the two need opposite fixes. Recall asks whether the right text is anywhere in the results. Ranking asks whether it is near the top. If the right text is missing from the results, you fix the search itself. If it is present but low, you add a second stage that reorders the results.
That second stage is a reranker. The first stage casts a wide net, say the top 50, and it has one job: do not miss. The reranker then scores each of those 50 against the question with a more accurate model and keeps the best 5 for the prompt. The rule to remember is: pass more, keep fewer.
Vector search uses a bi-encoder: a model that turns the question into one vector and, separately and ahead of time, each document into one vector, then compares the two. Because document vectors do not depend on the question, they are computed once and stored, which is why search over millions of documents takes milliseconds. But each vector is a compressed summary, and comparing two summaries catches shared topic more than whether the document really answers the question.
A reranker is usually a cross-encoder: a model that takes the question and one document joined into a single input and reads them together. Attention, the part of the model that relates words to each other, runs across both texts at once, and the model outputs one relevance score. It answers a sharper question: not "are these about similar things" but "does this document answer this question".
That is also why it is slow. The score depends on the question, so nothing can be computed in advance, and 50 candidates means 50 model passes at query time. Over a million documents that is hopeless; over the 50 survivors of the first stage it is a small, fixed cost. Between the two sit late-interaction models, which keep several vectors per document and compare them token by token, trading some speed for some accuracy.
In a measured lab run, the collection was 818 technical lessons written in one voice, so every wrong answer was a plausible neighbour. There were 20 questions, each with exactly one right lesson chosen before any search ran. Dense embedding search alone put the right lesson in the top 5 for 55 percent of the questions. Adding an LLM reranker over the dense top 20 raised that to 75 percent, the largest single change in the table.
The same run showed the limit. Two of the 20 questions had their right lesson outside the top 20 that the reranker reordered, one of them at position 72. No reranker can fix those, because it only reorders what it is given.
Twenty questions is a small sample, so read these numbers as a direction, not as a promise for your system. The same run also found that plain BM25 keyword scoring, with no model at all, reached 75 percent in the top 5 on its own, and that fusing keyword and dense results reached 85 percent in the top 10. Measure your first stage too, not only the reranker.
Reranking adds one model pass over the candidate set for every query. Depending on the model, the number of candidates and the hardware, that can be tens of milliseconds or a few hundred. It is often small next to the time the language model spends writing the answer, but time your own rerank call against your own generation time before you decide.
The main dial is how many candidates you rerank. Latency rises in a straight line with the candidate count, while quality rises and then flattens. Sweep the count against your own quality metric and choose the lowest value that clears your bar. Reranking the top 20 instead of the top 50 often keeps most of the gain for less time.
Measure the right thing at each stage. Recall at 50, whether the answer is anywhere in the first stage's 50, belongs to retrieval and barely moves when you add a reranker. Precision at 5, the share of the kept 5 that are relevant, is what a reranker moves. Order-aware metrics such as mean reciprocal rank (MRR) and nDCG suit a stage whose only job is order. If the top results are near-copies of one fact, a diversity pass such as Maximal Marginal Relevance (MMR) can follow the rerank.
Before you add one, pull the raw top 50 for a handful of failing questions and read them. If the right chunk is there but low, reranking is the fix. If it is missing, the problem is upstream, in chunking, the embedding model or the query, and a reranker will not help. If the answers already use the right chunks, a reranker only adds time.
There are three common ways to get one. A hosted API such as Cohere Rerank needs nothing to run, but adds a per-call fee and sends your documents to a vendor. An open cross-encoder such as BAAI's bge-reranker-v2-m3 runs on your own hardware with no per-query fee. A general cross-encoder from the sentence-transformers library, trained on a passage-ranking dataset such as MS MARCO, makes a free baseline.
Keep the final list short. In a separate measured run with the small model qwen3:4b, showing it the top 20 retrieved documents instead of the top 5 raised the chance that the right one was present from 11 to 18 of 20 questions, but cut the questions where it picked the right one from 10 to 6. A reranker lets you keep a small k and still have the right document near the top.
Cohere Rerank
A hosted rerank endpoint that takes a query and a list of documents and returns each document with a relevance score, sorted.
BAAI bge-reranker-v2-m3
An open cross-encoder reranker on Hugging Face that can be downloaded and run locally, with multilingual support.
sentence-transformers CrossEncoder
The sentence-transformers library provides a CrossEncoder class and pretrained rerankers trained on MS MARCO passage ranking.
ColBERT
A late-interaction retrieval model that keeps one vector per token and scores query and document by matching those token vectors.
A bi-encoder turns the question and each document into separate vectors and compares them, so document vectors can be stored in advance and search is fast. A cross-encoder reads the question and one document together and outputs a relevance score, which is more accurate but must run once per candidate at query time.
No. It only reorders the candidates the first stage returned, so recall at the candidate depth stays the same. It improves precision in the few results you keep. If the right document was never retrieved, fix the first stage instead.
A common pattern is to retrieve about 50 and keep about 5, but measure it. Latency grows in a straight line with the candidate count while quality flattens, so choose the smallest count that clears your quality bar; often 20 keeps most of the gain.
One model pass over the candidates per query, from tens of milliseconds to a few hundred depending on the model, candidate count and hardware. It is often small next to answer generation, but time both in your own system.
A hosted API such as Cohere Rerank is the fastest way to find out whether reranking helps, with nothing to run, but it has a per-call fee and your data leaves your systems. An open model such as a BGE reranker removes the fee and keeps data in house, but you run the model yourself.
RAG Caching and Cost: Semantic Cache, Embedding Reuse, and the Bill
RAG triples a chat bill through retrieved context. Cut it with a semantic cache, embedding reuse, and a reranker, without serving confidently wrong answers.
Design a RAG System at Scale
System design of a production RAG system. Covers chunking, embeddings, vector search, hybrid retrieval, reranking, caching, evaluation and cost.
Document Ingestion: The Layer That Quietly Caps Everything
Retrieval quality cannot exceed the quality of the text you extracted. Ingestion is the true ceiling of a RAG system, upstream of chunking, reranking, and hybrid search.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
Hybrid Retrieval
Running dense vector search and sparse keyword search together and fusing the results, so exact terms like an error code and paraphrased meaning both count.
Agentic RAG
Agentic RAG is RAG where a language model runs the search in a loop. It decides whether to search, what to search for and where. Then it checks if the text it found answers the question.
ANN (Approximate Nearest Neighbor)
Search that finds vectors close to a query without scanning every item, trading a small chance of missing a true neighbor for a large gain in speed. HNSW and IVF are the common indexes.
HNSW
Hierarchical Navigable Small World, a graph-based nearest neighbor index that gives high recall at low latency by navigating layers of links, at the cost of more memory than IVF.
This page explains the idea. The "Reranking: The Highest-Return Change in Your RAG Stack" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 210 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Reranking: The Highest-Return Change in Your RAG Stack lessoncourse 2
Take models from notebook to production, with labs on real models.