Query Transformation and Routing: Fixing Retrieval Before It Runs

Two strings.

The user types: can I get my money back?

The chunk that answers them says: Refund eligibility: purchases made through the self-serve checkout are refundable within 30 days of the transaction date.

Same meaning. Almost no shared vocabulary, and a completely different register: one is a casual first-person question, the other is third-person policy prose. Your embedding model maps text into a space where wording and style shape position, so these two land far enough apart that a chunk about updating payment methods, which happens to share the casual FAQ register and the word "money," can outscore the actual refund policy. The user gets an answer about credit cards. The refund chunk was in the index the whole time, cleanly chunked, well embedded, correctly ranked among its own kind.

Everything else in this chapter so far has worked on the index side of the equals sign. You cut better chunks, added a reranker, ran BM25 next to the embeddings, cleaned up extraction. All of that improves what the corpus looks like. None of it touches the other input to the similarity function: the query itself. The hybrid retrieval lesson already met one narrow version of this problem, the exact-token query that embeddings blur, and fixed it with a second index. This lesson is about the broad version: the query whose meaning is right but whose wording, register, or scope points the search at the wrong neighborhood. That gets fixed by changing the query before retrieval runs, and by sometimes deciding that retrieval should not run at all.

The reason this deserves a full lesson is leverage. A retrieval miss can come from either input to the similarity function, but the two inputs cost wildly different amounts to fix. The corpus side is millions of documents; the query side is a few dozen tokens at request time. When the cheap side is also the one you have not touched, that is where the next percentage points of recall live.