Earlier in this series, I covered:
This article takes the next step: how a reranker reorders the retrieved passages before they reach the language model.
Initial retrieval has to search a large collection. It may use sparse, dense, or hybrid signals to return a manageable set of candidate passages. A reranker operates after that search. It examines the smaller candidate set again and produces a new relevance order.
That second stage can use a more detailed query–passage comparison than the initial retrieval stage. But its scope is narrower: a reranker cannot promote a passage that was never retrieved.
Retrieval and reranking are separate stages
A basic retrieval system ranks chunks directly from an index and passes the highest-ranked results onward. A two-stage system introduces another step. The first stage retrieves a broader candidate list. The second stage scores or orders only those candidates again.
The two stages solve different operational problems. Initial retrieval must search efficiently across the indexed collection. Reranking has fewer passages to inspect, so it may compare the query and each candidate together in more detail.
The output of reranking is normally a reordered list and, depending on the system, a smaller selected subset. The underlying passages do not change. Their position and selection for the next stage may change.
What a reranker receives and returns
At minimum, a text reranker receives the user query and a list of candidate passages. It evaluates each query–passage relationship and returns the candidates in a new order, often with a new relevance value for each item.
Those values belong to the reranking stage. They should not automatically be interpreted as probabilities, compared with scores from a different model, or assumed to have the same meaning as the initial retrieval scores. The important output is the resulting order and the passage identities attached to it.
A reranker may return every candidate in a new order or only the first portion of the reordered list. The application then decides which passages continue into context assembly. That decision is distinct from the reranker’s act of assigning relevance.
Common reranking mechanisms
A cross-encoder is a common reranking pattern. Instead of creating one reusable representation for the query and another for each passage, it processes a query–passage pair together and produces a relevance value for that pair. Repeating the comparison across the candidate list creates a new ranking.
Other systems may use a language model to judge whether each passage addresses the query, or combine model scores with rules and structured signals. Some search platforms expose reranking as a built-in second-stage retriever. The implementation varies, but the pipeline boundary remains similar: the reranker operates on results already produced by retrieval.
A reranker is therefore not a replacement for indexing or first-stage retrieval. Applying a detailed pairwise model across an entire large corpus would be a different and usually more expensive operation than applying it to a bounded candidate set.
Two examples of reranking
Consider a fictional support assistant. The user asks, “Can I use a recovery code after replacing my sign-in phone?” Initial retrieval returns four passages. A passage about generating recovery codes ranks first because it shares several exact terms. A passage explaining recovery-code use during phone replacement ranks third.
A reranker compares the complete query with each candidate. It may move the phone-replacement passage above the code-generation passage because it addresses the full situation rather than only the shared terminology. The passage texts remain unchanged; their order changes.
Now consider a fictional travel-policy assistant. The question is, “Can a contractor claim meals during same-day travel?” A general meal-allowance passage contains many matching words, while a contractor-eligibility passage uses different wording but establishes who may claim the allowance.
Reranking may move the eligibility passage upward because it is necessary to answer the specific question. It still cannot establish whether the policy is current, resolve a contradiction between sources, or recover an eligibility passage omitted by initial retrieval.
Reranking cannot repair every retrieval problem
The strongest limitation is candidate-set dependence. If initial retrieval misses the needed passage, the reranker never sees it. Reranking can reorder the supplied candidates; it cannot search beyond them unless the application runs another retrieval step.
Input length can also matter. A reranking model may truncate or divide long passages, which can hide the relevant portion or change how the passage is evaluated. Implementations handle this differently.
Reranking also adds computation and another model or service boundary. That may affect latency, cost, observability, and failure handling, although the impact depends on the model and system design.
Finally, relevance is not correctness. A highly ranked passage may still be stale, factually wrong, contradictory, or insufficient. Metadata filtering, source freshness, authorization, retrieval coverage, and faithful generation remain separate concerns.
Initial retrieval searches the index and produces a candidate set. Reranking performs a second relevance assessment over that smaller set and may change which passages appear first or continue to model input.
A cross-encoder commonly scores the query together with each candidate passage, but other model- and rule-based mechanisms exist. The stable distinction is the stage boundary, not one particular implementation.
Reranking cannot recover an unseen passage, validate a source, or guarantee a grounded answer. It produces another relevance ordering over the candidates it receives.
After reranking, selected passages still have to be assembled into the language model’s input. The next explainer will cover that transition: what context assembly adds, how passage boundaries and source labels may be represented, and why model input is not simply a copy of the retrieval result.