VereskLearn

Module 4 · Retrieval

Lesson 4.1

Vector search

Vector search turns the question into numbers and ranks every chunk by how close its numbers are.

Lesson 20 of 622 min read

The problem

Why it matters

Without it

A keyword search needs the same words. A question about "money back" finds nothing in a chunk that says "refund".

With it

Each chunk and the question become lists of numbers. The search ranks chunks by how close their lists are, so "money back" finds "refund".

How it works

  • Each chunk is stored with a vector

    A vector is a list of numbers that stands for the meaning of a chunk. Each chunk gets its vector once, when the documents are loaded. The example has three chunks, and each vector has three numbers.

    ChunkVector
    Refunds take 5 to 7 days.0.9, 0.1, 0
    Use Forgot password to reset.0, 0.9, 0.2
    Orders ship in 2 days.0.1, 0.1, 0.9
    The three FAQ chunks and their vectors, as stored before any question arrives.
    // Three chunks, each with the vector stored at load time.const chunks = [  { text: "Refunds take 5 to 7 days.", vector: [0.9, 0.1, 0] },  { text: "Use Forgot password to reset.", vector: [0, 0.9, 0.2] },  { text: "Orders ship in 2 days.", vector: [0.1, 0.1, 0.9] },];for (const c of chunks) console.log(c.vector, c.text);

    Output

    [ 0.9, 0.1, 0 ] Refunds take 5 to 7 days.
[ 0, 0.9, 0.2 ] Use Forgot password to reset.
[ 0.1, 0.1, 0.9 ] Orders ship in 2 days.

  • The question gets a vector the same way

    The question goes through the same embedding process at search time. The words "money back" become the vector 0.85, 0.15, and 0 in this example. The plot puts each vector at its first number on the x axis and its second on the y axis.

    RefundsPasswordOrdersQuestion
    Each chunk and the question, plotted by their first two numbers. The question sits closest to the refund chunk.

  • The dot product scores each chunk

    The dot product multiplies matching numbers in the two lists, then adds the products. For the refund chunk it is 0.85 times 0.9, plus 0.15 times 0.1, plus 0 times 0, which is 0.78. The password chunk scores 0.14 and the orders chunk scores 0.10.

    Scores for the question "money back", one bar per chunk. A higher score means a closer match.
    // Multiply matching numbers, then add the products.function dot(a: number[], b: number[]): number {  return a.reduce((sum, value, i) => sum + value * b[i], 0);}const question = [0.85, 0.15, 0];const refund = [0.9, 0.1, 0];console.log(dot(question, refund).toFixed(2));

    Output

    0.78

  • Sort the scores and keep the top k

    Sort the scores from high to low, then keep the first k of them. With k set to 2, the refund chunk and the password chunk come back. The orders chunk, at 0.10, is dropped.

    • Refunds take 5 to 7 days.0.78, returned
    • Use Forgot password to reset.0.14, returned
    • Orders ship in 2 days.0.10, dropped
    Results for "money back" with k set to 2. The highlighted chunks are returned.
    // Sort the scores from high to low, then keep k.const scored = [  { text: "Refunds take 5 to 7 days.", score: 0.78 },  { text: "Use Forgot password to reset.", score: 0.14 },  { text: "Orders ship in 2 days.", score: 0.1 },];const k = 2;const top = scored.sort((a, b) => b.score - a.score).slice(0, k);console.log(top.map((t) => t.text));

    Output

    [ 'Refunds take 5 to 7 days.', 'Use Forgot password to reset.' ]

  • Weak matches still come back

    A question with no good match still gets k chunks. The question "gift cards" has the vector 0, 0, and 0.1 in this example. The best score is only 0.09, so the results are weak, yet two chunks still come back. Real systems use a vector index to skip most chunks, but this example scores every chunk.

    Scores for "gift cards". No chunk is about gift cards, and the top score is only 0.09.

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Assuming a high score proves the answer is right. The score only shows how close two lists are.
  • Comparing vectors from two different models. Their numbers do not line up, so the scores mean nothing.
  • Setting k too low. The right chunk may rank third and never reach the model.

Use it when and skip it when

Is it the right tool?

Use it when

  • Users ask in words that differ from the document, such as "money back" for "refund".
  • The documents are short and mostly prose, such as help articles or manuals.

Skip it when

  • The question names an exact code, such as an order number. Use an exact lookup or keyword search.
  • The answer depends on an exact field, such as a version number. Use a metadata filter first (lesson 4.2).

Words to know

Terms in this lesson

Chunk
A small part of a document, stored and searched on its own.
Score
A number that shows how well a chunk matches the question.
Dot product
Multiply matching numbers in two lists, then add the products.
Top k
The k highest-scoring results, where k is a number you choose.
Full code