Reranking
What is it?
Reranking is a second pass in search that reorders an initial batch of results using a more accurate model, so the best matches rise to the top.
Explain like I'm 5
Why was it created?
Fast first-stage retrieval (like vector search) is good at recall but rough on precise ordering. Reranking was added to sharpen the top results without slowing the initial search.
Where is it used?
- RAG pipelines
- Document and product search
- Question answering
- Recommendation ranking
Why should developers care?
In RAG and search, what lands in the top few results decides answer quality — reranking is often the cheapest big win for relevance.
How does it work?
A fast retriever returns, say, the top 50 candidates. A heavier cross-encoder model then scores each candidate against the query directly and reorders them, so the final top 5 handed to the user or LLM are the most relevant.
Real-world example
A RAG app retrieves 50 chunks by vector similarity, reranks them, and passes only the 5 best to the LLM — improving the answer without enlarging the prompt.
Common use cases
- Improving RAG answer quality
- Sharpening search result order
- Filtering noisy retrieval
- Prioritizing which context to feed an LLM
Advantages
- Big precision gains at the top
- Reuses fast first-stage retrieval
- Reduces irrelevant context to the LLM
- Tunable candidate count
Disadvantages
- Adds latency and cost per query
- Cross-encoders are heavier to run
- Limited by first-stage recall
- Another component to maintain
When should you use it?
When first-stage retrieval returns roughly right results but the ordering or top-k precision isn't good enough.
When should you avoid it?
When latency is critical and first-stage results are already precise, or when the corpus is tiny.
Alternatives
Related terms
Interview questions
Beginner
- What does a reranker do to search results?
- Why not just return the retriever's order?
Intermediate
- How does a cross-encoder differ from the embedding retriever?
- Why rerank a large candidate set down to a few?
Senior
- How does reranking interact with first-stage recall limits?
- What latency/quality trade-offs govern candidate count?
Common misconceptions
- "Reranking fixes bad retrieval" — it can't surface what the first stage never retrieved.
- "It replaces vector search" — it complements it as a second stage.
Fun facts
- Rerankers are usually cross-encoders that read query and document together, unlike bi-encoder retrievers.
- Reranking often improves RAG more than swapping the LLM does.
Timeline
- 2019 — Cross-encoder reranking popularized for neural search
- 2020s — Standard stage in RAG pipelines
Learning resources
Quick summary
Reranking reorders an initial batch of retrieved results with a more accurate model so the best matches end up on top.
Cheat sheet
- Second-stage result reorder
- Cross-encoder scores query+doc
- Boosts RAG/search precision
- Bounded by first-stage recall