VereskLearn

Module 4 · Retrieval

Lesson 4.5

Reranking

Reranking reads each candidate together with the question, then sorts the candidates again more carefully.

Lesson 24 of 622 min read

The problem

Why it matters

Without it

A fast first search compares the question and each chunk separately. Loose matches can look alike, so the best answer may sit below a weaker one.

With it

A slower model reads the question and each candidate as one input. It scores how well the candidate answers, and the best ones move to the top.

How it works

  • The first search returns a short list

    The first search returns more candidates than the answer needs, such as 20. This example keeps four candidates, in the order the first search gave them. That order is rough, so the best candidate may sit lower in the list.

    • Refunds take 5 to 7 days after the item arrivesposition 1
    • Order status is shown on the account pageposition 2
    • Refunds for gift cards are not possibleposition 3
    • Shipping takes 2 business days in most areasposition 4
    Four candidates in first-pass order, from the fast search.
    // Four candidates in the order the fast search returned them.const candidates = [  "Refunds take 5 to 7 days after the item arrives.",  "Order status is shown on the account page.",  "Refunds for gift cards are not possible.",  "Shipping takes 2 business days in most areas.",];console.log(candidates.length, "candidates");

    Output

    4 candidates

  • A reranker reads the question with each candidate

    The reranker takes the question and one candidate as a single input. It returns one relevance score for that pair, so the score reflects both texts. Each pair needs its own model run, so only a short list is reranked.

    1. Question
    2. One candidate
    3. Reranker model
    4. Relevance score
    One pair goes through the reranker to produce one score.

  • The fake scorer counts shared words

    The fake scorer counts the question words that appear in the candidate. The question "how long do refunds take" has five words, and the top candidate shares "refunds" and "take". That gives 2 of 5, or 0.40, while a real reranker is a trained model.

    Fake scores for "how long do refunds take". Each score is shared words divided by 5.
    // Count the question words found in the candidate.const question = ["how", "long", "do", "refunds", "take"];const candidate =  "refunds take 5 to 7 days after the item arrives.";const words = candidate.split(" ");const shared = question.filter((w) => words.includes(w));console.log(shared, (shared.length / question.length).toFixed(2));

    Output

    [ 'refunds', 'take' ] 0.40

  • Sort and keep the top n

    The candidates are sorted by the new score, highest first. Keeping the top n, here 2, sends only two passages to the answer model. The order from the first search no longer matters after this sort.

    • Refunds take 5 to 7 days after the item arrives.0.40, kept
    • Refunds for gift cards are not possible.0.20, kept
    • Order status is shown on the account page.0.00, dropped
    • Shipping takes 2 business days in most areas.0.00, dropped
    The two highest scores are kept. The other two are dropped.
    // Sort by the new score, highest first. Keep the top n.const scored = [  { passage: "Order status", score: 0 },  { passage: "Refunds take 5 to 7 days", score: 0.4 },  { passage: "Refunds for gift cards", score: 0.2 },];const top = scored.sort((a, b) => b.score - a.score).slice(0, 2);console.log(top.map((t) => t.passage));

    Output

    [ 'Refunds take 5 to 7 days', 'Refunds for gift cards' ]

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Reranking too few candidates. The reranker cannot promote a chunk that the first search dropped.
  • Reranking a long list for every query. Each pair costs a model call, so the cost grows with the list.
  • Trusting the new order without a test. Check it on the golden questions (lesson 2.1).

Use it when and skip it when

Is it the right tool?

Use it when

  • The first search returns the right chunk, but ranks it low.
  • Answers must be precise, and a few extra model calls per question are acceptable.

Skip it when

  • The first search already ranks the answer first on most test questions.
  • The time budget is tight. Each pair adds delay before the answer starts.

Words to know

Terms in this lesson

Candidate
A chunk returned by the first search, waiting to be reranked.
Reranker
A model that reads a question and a passage together and gives a relevance score.
Cross-encoder
A reranker that reads the question and the passage as one input.
Latency
The time from sending a request to getting the response.
Full code