Reranking: Improving Relevance

Objectives

• Understand why reranking improves results
• Implement a reranker using a cross-encoder
• Integrate reranking into a RAG pipeline

The Bi-Encoder Problem

All the search you’ve built so far relies on a bi-encoder: the query and documents are encoded separately, on their own, and compared via dot product. This is what makes search instantaneous across a million chunks, since the corpus vectors were computed once and for all. It’s also what makes it imprecise, because the model never sees the query and document together: it must summarize each text into a vector without knowing what will be asked of it later.

A cross-encoder, also called a reranker, takes the (query, document) pair as input and produces a much finer relevance score, because it reads both texts jointly and can detect that a document discusses the topic but in an opposing sense. This gain comes at a cost: the model must process each document individually, which prohibits using it on the entire corpus. The solution is to employ it where it’s cost-effective, on a small set of candidates already shortlisted.

Hence the two-stage pipeline, where each model does what it does best: the fast one narrows the field, the precise one makes the final cut.

Query → Bi-encoder → Top 50 Candidates → Cross-encoder → Top 5 Results

Reranking with Cohere

Cohere exposes a high-performance reranker via API, which avoids hosting a model. The following function chains the two steps: it first retrieves twenty candidates via vector similarity, then asks rerank-v3.5 to keep only five. Note the index_original field preserved in the result: the reranker returns positions relative to the candidate list, and you’ll need this to map back to the actual document in your corpus.

import cohere
from openai import OpenAI

co = cohere.Client(api_key="your-cohere-key")
openai_client = OpenAI()

def search_and_rerank(
    question: str,
    documents: list[str],
    embeddings_corpus,
    k_retrieval: int = 20,
    k_final: int = 5
) -> list[dict]:
    """Semantic search + reranking."""
    # Step 1: Fast retrieval (bi-encoder)
    query_emb = openai_client.embeddings.create(
        input=question,
        model="text-embedding-3-large"
    ).data[0].embedding

    import numpy as np
    scores = embeddings_corpus @ np.array(query_emb)
    top_indices = np.argsort(scores)[-k_retrieval:][::-1]
    candidates = [documents[i] for i in top_indices]

    # Step 2: Reranking (cross-encoder)
    rerank_response = co.rerank(
        query=question,
        documents=candidates,
        top_n=k_final,
        model="rerank-v3.5"
    )

    return [
        {
            "text": candidates[r.index],
            "rerank_score": r.relevance_score,
            "original_index": int(top_indices[r.index])
        }
        for r in rerank_response.results
    ]

Reranking with an LLM

If you prefer not to add another provider to your architecture—due to contractual reasons, data localization, or simply billing—an LLM you already call can fill the role. You submit the question and a document, ask for a score from 0 to 10, and sort. Temperature set to zero and max_tokens reduced to five ensure deterministic, cheap output; the try/except handles cases where the model responds with something other than a number.

from openai import OpenAI

client = OpenAI()

def llm_reranker(
    question: str,
    candidates: list[str],
    k: int = 5
) -> list[dict]:
    """Uses an LLM to score relevance."""
    scores = []

    for i, doc in enumerate(candidates):
        response = client.chat.completions.create(
            model="gpt-5.6-terra",
            messages=[{
                "role": "user",
                "content": (
                    f"On a scale of 0 to 10, how relevant is this document "
                    f"for answering the question?\n\n"
                    f"Question: {question}\n\n"
                    f"Document: {doc}\n\n"
                    f"Respond only with a number between 0 and 10."
                )
            }],
            temperature=0,
            max_tokens=5
        )
        try:
            score = float(response.choices[0].message.content.strip())
        except ValueError:
            score = 0.0
        scores.append({"index": i, "text": doc, "score": score})

    scores.sort(key=lambda x: x["score"], reverse=True)
    return scores[:k]

This version costs one call per candidate, meaning twenty calls and as much cumulative latency for a single user question. In practice, this is prohibitive in an interactive interface. Instead, we ask the model to rank all documents at once, presenting each candidate truncated to 200 characters and numbered, expecting only a list of indices in return.

def llm_batch_reranker(
    question: str,
    candidates: list[str],
    k: int = 5
) -> list[int]:
    """Reranking in a single LLM call."""
    formatted_docs = "\n".join(
        f"[{i}] {doc[:200]}" for i, doc in enumerate(candidates)
    )

    response = client.chat.completions.create(
        model="gpt-5.6-terra",
        messages=[{
            "role": "user",
            "content": (
                f"Rank these documents by relevance to the question. "
                f"Return the indices of the {k} most relevant "
                f"documents, separated by commas, from most to "
                f"least relevant.\n\n"
                f"Question: {question}\n\n"
                f"Documents:\n{formatted_docs}\n\n"
                f"Indices (e.g., 3,1,7,0,5):"
            )
        }],
        temperature=0,
        max_tokens=50
    )

    indices_str = response.choices[0].message.content.strip()
    return [int(i.strip()) for i in indices_str.split(",")][:k]

Integration into the RAG Pipeline

In a complete pipeline, reranking fits between retrieval and generation without changing anything else. The key point in the class below is the 0.3 threshold on relevance_score: it allows returning fewer than five contexts when no candidate is truly relevant, rather than feeding the generator noise. Often this filter, more than the re-ranking itself, eliminates hallucinated answers.

class RAGPipelineWithReranking:
    def __init__(self):
        self.client = OpenAI()
        self.co = cohere.Client()

    def respond(self, question: str, chunks, embeddings_corpus):
        # 1. Broad retrieval
        candidates = self.search(question, embeddings_corpus, k=20)

        # 2. Reranking
        candidate_docs = [chunks[i]["text"] for i in candidates]
        reranked = self.co.rerank(
            query=question,
            documents=candidate_docs,
            top_n=5,
            model="rerank-v3.5"
        )

        # 3. Select the best
        contexts = [
            chunks[candidates[r.index]]
            for r in reranked.results
            if r.relevance_score > 0.3  # Relevance threshold
        ]

        # 4. Generate the answer
        return self.generate(question, contexts)

Impact of Reranking

These figures tell precisely what a reranker does. Recall doesn’t move, and that’s logical: the candidates are the same, and the cross-encoder cannot rescue a document the bi-encoder didn’t surface. However, precision and MRR improve significantly, because the right document moves from sixth place to first. Practical corollary: if your recall is poor, reranking won’t save you; you must first broaden retrieval or revisit chunking.

Summary

• Reranking re-ranks candidates with a more precise model
• Typical pipeline: bi-encoder (top 20–50) → cross-encoder (top 5)
• Cohere rerank-v3.5 is a turnkey solution
• An LLM can serve as a reranker if you prefer to avoid a dependency
• The precision gain is significant (typically 20–35%)