What Is Reranking?
Also known as: re-ranking, cross-encoder reranker, reranker
Definition
Reranking is a second retrieval stage in RAG where a more precise model re-scores the top candidate passages from the first search and reorders them, so the LLM receives only the most relevant context.
Reranking Explained
First-stage retrieval has to be fast, so it compares one query vector against millions of stored vectors. That is efficient but approximate. A reranker takes the top 20 to 100 candidates and reads each one together with the question, typically using a cross-encoder model, and produces a sharper relevance score. The best handful go to the LLM.
The cost trade-off is clear: the reranker is slower per item but only runs on a short list. The gain is largest when the corpus holds many near-duplicate passages, which is normal in manufacturing: dozens of work instructions that differ by one part number, revisions of the same procedure, or supplier specifications with identical wording.
Reranking is often combined with hybrid search, which merges keyword search (BM25) and vector search. Exact identifiers such as part numbers, drawing numbers and ECO numbers match poorly in embeddings but perfectly in keyword search, so hybrid retrieval followed by reranking is a common strong baseline.
Rerankers can run locally. Open models in the bge-reranker and similar families fit on modest GPUs, so the whole retrieval stack stays inside the customer network.
Why It Matters
- It raises answer accuracy without changing the LLM or re-indexing documents.
- It reduces the context sent to the LLM, which lowers latency and token cost.
- It helps most where documents are repetitive, as in controlled procedures and part catalogs.
- It can run on-prem, so it fits air-gapped deployments.
In Practice
A technician asks for the torque value for a hydraulic fitting on model X. Hybrid search returns 40 candidates from 9 similar work instructions. The reranker scores each against the full question and puts the correct step from the model X instruction first and a similar step from model Y fifth. Only the top 5 go to the LLM, which answers from the right document instead of blending two.
Frequently Asked Questions
Do I always need a reranker?
No. For a small, clean corpus, good chunking and hybrid search may be enough. Add a reranker when similar documents cause wrong or mixed answers.
What is the difference between a reranker and an embedding model?
An embedding model encodes query and passage separately for fast search. A cross-encoder reranker reads them together, which is more accurate but too slow to run over the whole corpus.
Does reranking help with part numbers?
Indirectly. Keyword search catches exact identifiers and the reranker orders the combined results. Treat part numbers as metadata filters where possible.
How much latency does it add?
Typically tens to a few hundred milliseconds for a short candidate list on a GPU, depending on the model and list size.
Related Terms
RAG (Retrieval-Augmented Generation)
RAG (Retrieval-Augmented Generation) is an architecture that searches your own document store for passages relevant to a question, inserts them into the LLM prompt, and instructs the model to answer only from that retrieved evidence.
Embedding
An embedding is a list of numbers, typically several hundred to a few thousand dimensions, that represents the meaning of a piece of text, image, or code so that semantically similar items produce vectors that sit close together.
Vector Database
A vector database is a system that stores high-dimensional embedding vectors and retrieves the ones most similar to a query vector, enabling semantic search where results match meaning rather than exact keywords.
Chunking
Chunking is the step in a RAG pipeline that splits long documents into smaller passages, usually a few hundred tokens each, so each can be embedded, indexed and retrieved on its own. Chunk boundaries strongly affect answer quality.
Semantic Search
Semantic search retrieves results by meaning rather than exact word match, using embeddings to find text that is conceptually close to the query. A search for "supplier delivered damaged goods" can find records that say "vendor shipment arrived broken".
Working with Reranking in a live environment? Our engineers do this every day - and our AI agents automate most of it.
Try it here, free
Hybrid Search Cost Calculator
Estimate the monthly infrastructure cost of running vector search and keyword search together, including an optional reranker, based on your node counts and query volume.
Keep exploring
Free tools
- Hybrid Search Cost CalculatorEstimate the monthly infrastructure cost of running vector search and keyword search together, including an optional reranker, based on your node counts and query volume.
- Enterprise RAG Security ChecklistA practical control checklist covering entitlement-aware retrieval, corpus data governance, query security, model integrity, and audit for enterprise RAG systems.
- Private RAG Corpus Sizing CalculatorEstimate chunk counts, vector index storage, raw text volume, and embedding compute time before you build a private retrieval system over your document estate.
Guides
- Hybrid Search: Combining BM25 and Vector Retrieval for RAGHybrid search for RAG: why pure vector retrieval misses exact matches, how BM25 fixes it, and how reciprocal rank fusion combines both reliably.
- Enterprise RAG Architecture: A Practitioner's Blueprint for 2026A practitioner's blueprint for enterprise RAG in 2026: ingestion, chunking, embedding, retrieval, rerank, generation, and the eval loop that keeps it honest.
- Choosing an Embedding Model for Enterprise Search and RAGChoose the right embedding model for enterprise RAG: dimensions, multilingual support, and domain adaptation tradeoffs that determine retrieval quality.
Troubleshooting
- Why adding a reranker isn't improving your RAG results, and how to fix itA reranker that doesn't help usually means too few candidates are being reranked or the wrong model is being used. Fix the candidate pool size and model choice.
- Why RAG retrieves the wrong documents, and how to fix itRAG returns irrelevant chunks when the embedding model differs between indexing and query time, chunks are too large, or queries need rewriting. Fix it here.