Reranking

What Is Reranking?

Reranking is a second-stage retrieval step. A fast retriever first finds a larger set of candidates. A more precise model then scores those candidates again and changes their order.

Why Do We Need Reranking?

Initial vector retrieval is designed to be fast over a large collection. The highest-scoring candidates are useful, but their ordering is not always perfect.

Two-Stage Retrieval Pipeline

This design balances speed and relevance: retrieval narrows the search space, while reranking spends more computation only where it matters.

Bi-Encoder vs Cross-Encoder

What Does a Reranker Actually Score?

A reranker receives a query and a candidate document. It produces a relevance score used to reorder the candidate list.

The important point is that the reranker evaluates the relationship between the query and each candidate, rather than relying only on precomputed vector distance.

Candidate Set: Retrieve More, Rerank Fewer

You normally do not rerank an entire database. Instead, retrieve a manageable candidate set such as 20, 50, or 100 results and rerank those candidates.

Rerank-K vs Final Top-K

Retrieve-K is the broad candidate count. Final Top-K is what you finally pass to the next stage, such as an LLM.

Simple Python Reranking Example

The following example uses a small manual relevance function to demonstrate the idea. Real applications usually use a trained reranker model.

query = "reset my API key"
candidates = [
    "How to reset your API key",
    "How to update your billing address",
    "API rate limits and quotas",
]

def simple_score(query, document):
    query_words = set(query.lower().split())
    document_words = set(document.lower().split())
    return len(query_words & document_words)

ranked = sorted(
    candidates,
    key=lambda doc: simple_score(query, doc),
    reverse=True,
)

for document in ranked:
    print(simple_score(query, document), document)

This is only a teaching model. A production reranker learns richer relevance patterns than word overlap.

Using a Cross-Encoder in Python

A common teaching example is a cross-encoder that scores query-document pairs together.

from sentence_transformers import CrossEncoder

model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

query = "How do I reset my API key?"
documents = [
    "Reset your API key from the developer settings page.",
    "You can update your billing address from your account.",
    "API requests are limited by your account quota.",
]

pairs = [(query, document) for document in documents]
scores = model.predict(pairs)

ranked = sorted(
    zip(scores, documents),
    key=lambda item: item[0],
    reverse=True,
)

for score, document in ranked:
    print(round(float(score), 3), document)

The model name above is an example for learning. In production, choose and evaluate a reranker that fits your language, domain, quality, latency, and licensing requirements.

Reranking in a RAG Pipeline

Reranking can improve the context sent to the LLM by pushing highly relevant chunks toward the top and removing weaker candidates before generation.

Reranking and Metadata Filtering

Hard constraints such as region, product, language, or access permissions should normally be enforced rather than asking a reranker to ignore invalid documents.

Latency and Cost

When Reranking Helps Most

When Reranking May Not Be Worth It

Do not add a reranker because it sounds advanced. Add it when an evaluation set shows a measurable benefit.

Common Reranking Mistakes

How to Evaluate Reranking