VereskLearn

Module 1 · Talking to a model

Lesson 1.5

Cosine similarity

Cosine similarity is a score from -1 to 1 that shows how closely two lists point the same way.

Lesson 7 of 622 min read

The problem

Why it matters

Without it

Two embeddings are two long lists of numbers. Comparing them value by value gives no single clear answer.

With it

One score sums up the comparison. A high score means the texts are close in meaning, and a low score means they are not.

How it works

  • Both lists need the same length

    Cosine similarity compares two embeddings with the same number of values. The refund list is [0.9, 0.1, 0] and the money-back list is [0.8, 0.2, 0.1]. Both have three values, so each value pairs with the value in the same position.

    const refund = [0.9, 0.1, 0];const moneyBack = [0.8, 0.2, 0.1];console.log("Same length ->", refund.length === moneyBack.length);

    Output

    Same length -> true

  • Multiply the pairs and add them

    Multiply the two values in each position, then add the three products. The products are 0.72, 0.02, and 0, so the dot product is 0.74. The dot product grows with the size of both lists, so the next block corrects for it.

    PositionRefundMoney backProduct
    10.90.80.72
    20.10.20.02
    300.10
    Sum0.74
    Each position, its product, and the dot product at the bottom.
    const refund = [0.9, 0.1, 0];const moneyBack = [0.8, 0.2, 0.1];// Multiply each pair, then add the products.const dot = refund.reduce((s, v, i) => s + v * moneyBack[i], 0);console.log("Dot product ->", dot.toFixed(2));

    Output

    Dot product -> 0.74

  • Divide by the two lengths

    The length of a list is the square root of the sum of its squared values. The refund list has length 0.91 and the money-back list has length 0.83. Dividing the dot product by both lengths gives 0.74 divided by 0.91 times 0.83, which is 0.984.

    1. Dot product 0.74
    2. Divide by 0.91 times 0.83
    3. Score 0.984
    The three numbers that make the score.
    const refund = [0.9, 0.1, 0];const moneyBack = [0.8, 0.2, 0.1];// Math.hypot gives the length of each list.const a = Math.hypot(...refund);const b = Math.hypot(...moneyBack);const dot = refund.reduce((s, v, i) => s + v * moneyBack[i], 0);const score = dot / (a * b);console.log(a.toFixed(2), b.toFixed(2), score.toFixed(3));

    Output

    0.91 0.83 0.984

  • Reading the score

    A score of 1 means the two lists point the same way. A score of 0 means they meet at a right angle, with no shared direction. A score of -1 means they point in opposite directions. The refund and password lists score 0.108, close to 0, so they share little direction.

    Scores of the refund list against two others. The money-back list is close to 1.
    const refund = [0.9, 0.1, 0];const password = [0, 0.9, 0.2];const dot = refund.reduce((s, v, i) => s + v * password[i], 0);const a = Math.hypot(...refund);const b = Math.hypot(...password);console.log("Refund vs password ->", (dot / (a * b)).toFixed(3));

    Output

    Refund vs password -> 0.108

  • Only the direction counts

    Scaling a list does not change the score. The list [2, 4] is twice the list [1, 2], so both point the same way and score 1. Search systems sort texts by this score, so it is only as good as the embeddings behind it.

    [1, 2][2, 4]
    The lists [1, 2] and [2, 4] lie on one line through zero, so they point the same way.
    const cosine = (a: number[], b: number[]) => {  const dot = a.reduce((s, v, i) => s + v * b[i], 0);  return dot / (Math.hypot(...a) * Math.hypot(...b));};const result = cosine([1, 2], [2, 4]);console.log("[1, 2] vs [2, 4] ->", result.toFixed(3));

    Output

    [1, 2] vs [2, 4] -> 1.000

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Comparing lists of different lengths. The formula needs the same number of values in both lists.
  • Reading the score as a percent. A score of 0.8 is not 80 percent sure. It only measures closeness in direction.
  • Choosing a cutoff without testing it. The right cutoff depends on your data, so test it on real examples.

Use it when and skip it when

Is it the right tool?

Use it when

  • You rank texts by how close they are in meaning.
  • You compare two embeddings that came from the same model.

Skip it when

  • The lists have different lengths or come from different models. The score then means nothing.
  • You need the size of the gap between two values, not the direction. A distance measure fits better.

Words to know

Terms in this lesson

Cosine similarity
A score from -1 to 1 for how closely two lists point the same way.
Dot product
Multiply each pair of values, then add the products.
Length
The square root of the sum of the squared values. It is also called the norm.
Ranking
Sorting results from the best match to the weakest.
Full code