VereskLearn

Module 10 · Beyond retrieval

Lesson 10.2

Fine-tuning

Fine-tuning trains a model further on example pairs, so it learns one fixed format or style.

Lesson 61 of 623 min read

The problem

Why it matters

Without it

A long prompt repeats the same format rules in every request, and the rules still fail on some replies.

With it

Training on examples of the right format teaches the pattern, so each request can use a shorter prompt.

How it works

  • Each example is an input and an ideal output

    A training pair holds an input and the ideal output for it. For a format task, the question "Refund time?" pairs with a reply in one exact JSON shape. JSON is a text format for data written as named values. The set in this lesson has five pairs in that shape.

    QuestionIdeal reply
    Refund time?{"answer":"5 to 7 days"}
    Shipping time?{"answer":"2 business days"}
    Reset password?{"answer":"Forgot password"}
    Gift card limit?{"answer":"not in the FAQ"}
    Change address?{"answer":"Account settings"}
    The five training pairs in this lesson.
    const pairs = [  { q: "Refund time?", a: '{"answer":"5 to 7 days"}' },  { q: "Shipping time?", a: '{"answer":"2 business days"}' },  { q: "Reset password?", a: '{"answer":"Forgot password"}' },  { q: "Gift card limit?", a: '{"answer":"not in the FAQ"}' },  { q: "Change address?", a: '{"answer":"Account settings"}' },];for (const p of pairs) console.log(p.q, "->", p.a);

    Output

    Refund time? -> {"answer":"5 to 7 days"}
Shipping time? -> {"answer":"2 business days"}
Reset password? -> {"answer":"Forgot password"}
Gift card limit? -> {"answer":"not in the FAQ"}
Change address? -> {"answer":"Account settings"}

  • Cleaning removes bad pairs

    Remove duplicates, wrong replies, and personal data before training. A pair whose reply says "5 to 7 dys" would teach a typo, so it is dropped. A bad example teaches a bad habit, because the model copies the errors it is trained on.

    Kept

    • Refund time? -> 5 to 7 days

    Dropped

    • Refund time? -> "5 to 7 dys"
    • Reason: typo in the reply
    One pair kept and one pair dropped during cleaning.

  • The split holds back test rows

    The rows are cut at 80 percent. The first 80 percent train the model, and the last 20 percent are kept for testing. With five rows, four train the model and one is held back. The test rows are never used for training, so the score stays honest.

    UseRowsQuestions
    Training4Refund time, Shipping time, Reset password, Gift card limit
    Test1Change address
    The split of the five rows at 80 percent.
    const rows = [  "Refund time?",  "Shipping time?",  "Reset password?",  "Gift card limit?",  "Change address?",];const cut = Math.floor(rows.length * 0.8);const train = rows.slice(0, cut).length;console.log("Train:", train, "Test:", rows.length - cut);

    Output

    Train: 4 Test: 1

  • Each pair becomes one JSON line

    A training file holds one JSON object per line. This layout is common because each line can be read alone. Each object holds a user message and an assistant message. The assistant message is the ideal reply, and its quotes are escaped inside the line.

    1. User message
    2. Assistant message
    3. One JSON line
    How one pair becomes one line of the training file.
    const row = JSON.stringify({  messages: [    { role: "user", content: "Refund time?" },    { role: "assistant", content: '{"answer":"5 to 7 days"}' },  ],});console.log(row);

    Output

    {"messages":[{"role":"user","content":"Refund time?"},{"role":"assistant","content":"{\"answer\":\"5 to 7 days\"}"}]}

  • The training run changes the weights

    The training run changes the model weights, which are the numbers inside the model, using the training rows. Training costs money and time, so start with a small set, such as 100 pairs, before using 10,000. Fine-tuning is a weak place to store facts that change, such as prices.

    Pairs in each training run, from a small first run to a full one.

  • The held-out test decides

    Run the new model on the test rows, and compare its replies with the old model. For example, it must beat the old model on 20 test pairs before it ships. Re-test the whole eval set, not only the new format, because training can change other answers.

    Old model

    • Same 20 test pairs
    • Score recorded before the change

    New model

    • Same 20 test pairs
    • Must score higher to ship
    The same test pairs, scored for two models.

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Using too few examples. A handful of pairs may not change the model enough, so test before you rely on it.
  • Testing on training pairs. The score then looks high and proves nothing.
  • Training on a wrong or private example. The model repeats it, so review each pair first.

Use it when and skip it when

Is it the right tool?

Use it when

  • The output must follow one fixed format or style, and prompts do not hold it.
  • You have many good examples, and real requests look like them.

Skip it when

  • The problem is missing facts. Training does not store new facts reliably.
  • You have only a few good examples, so you cannot test the result.

Words to know

Terms in this lesson

Training pair
One input with the ideal output for it.
JSON Lines
A file with one JSON object on each line.
Held-out set
Examples kept out of training and used only for testing.
Fine-tuning
Extra training on example pairs that changes the model weights.
Full code