VereskLearn

Module 5 · Writing the answer

Lesson 5.1

Building the context

The context is the text the model reads: the labeled sources, the rules, and the question.

Lesson 32 of 622 min read

The problem

Why it matters

Without it

Every retrieved chunk is pasted in, in no order and with no size limit. The text can grow past what the model accepts.

With it

The best chunks come first and each has an id. The total stays under a limit, so the model reads the sources it needs.

How it works

  • Each source gets a label

    Each retrieved chunk gets an id in square brackets, such as [faq-12]. The model copies that id when it names a source. Lesson 4.1 covers the search that finds these chunks.

    Labeled lineLength
    [faq-12] Refunds take 5 to 7 days.34
    [faq-40] Gift cards never expire.33
    [manual-3] Refunds need the order number.41
    Three labeled lines. Length counts every character, the id included.
    // One chunk, labeled with its id.const chunk = { id: "faq-12", text: "Refunds take 5 to 7 days." };const line = "[" + chunk.id + "] " + chunk.text;console.log(line);console.log("Length: " + line.length);

    Output

    [faq-12] Refunds take 5 to 7 days.
Length: 34

  • Lengths add up to a running total

    The context builder adds each line length to a running total. Before a line is kept, the new total must stay at or under 100 characters. The first two lines reach 67, and the third would bring the total to 108.

    Running total after each line, against a limit of 100 characters.

  • The loop stops at the first line that does not fit

    The loop keeps a line only when the total stays within the limit. It stops at the first line that would pass the limit, so every later line is skipped. Here faq-12 and faq-40 are kept, and manual-3 is left out.

    • [faq-12]34, kept
    • [faq-40]33, kept
    • [manual-3]41, left out
    Each line, its length, and whether the limit kept it.
    const maxCharacters = 100;const lengths = [34, 33, 41];let total = 0;let kept = 0;for (const length of lengths) {  if (total + length > maxCharacters) break;  total += length;  kept++;}console.log("Kept " + kept + " lines, " + total + " characters");

    Output

    Kept 2 lines, 67 characters

  • The order decides which lines survive

    The search ranks lines with the best match at the top, so the limit keeps the best ones. A different order keeps a different set of lines. Ranking must therefore happen before the limit is applied.

    Best match first

    • faq-12, 34: kept
    • faq-40, 33: kept
    • manual-3, 41: left out

    Longest first

    • manual-3, 41: kept
    • faq-12, 34: kept
    • faq-40, 33: left out
    The same three lines in two orders, with a limit of 100 characters.

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Sending every chunk for safety. More text costs more and can push the best chunk out of the limit.
  • Counting lines instead of size. Chunks differ in length, so measure the characters or tokens.
  • Dropping the ids. Without an id, the model cannot name the source of a fact.

Use it when and skip it when

Is it the right tool?

Use it when

  • The answer must come from retrieved passages, and the passages do not fit in one request.
  • You need the model to name the source of each fact.

Skip it when

  • The whole source text is short enough to send in full.
  • The question has one exact answer that a lookup can return.

Words to know

Terms in this lesson

Context
All the text sent to the model for one request.
Chunk
A small part of a document, stored and searched on its own.
Id
A short label that names one chunk, such as faq-12.
Token
A small piece of text, the unit that a model counts and bills.
Full code