VereskLearn

Module 3 · Preparing documents

Lesson 3.4

Chunking

Chunking splits a long document into small parts, so each part can be searched and quoted on its own.

Lesson 17 of 622 min read

The problem

Why it matters

Without it

A 40-page manual is one search item. Search returns the whole manual, and the model reads far more text than it needs.

With it

Each chunk is a small, self-contained part. Search returns the chunk that answers the question, so the citation is precise.

How it works

  • Start chunks at natural breaks

    Prefer paragraph and heading breaks, because they follow the structure the author wrote. A heading such as "Shipping" marks a natural start for a chunk. A cut at a fixed count can land in the middle of a sentence.

    Cut at a fixed count

    • Can land mid-sentence
    • A chunk may mix two topics

    Cut at a heading

    • Each chunk starts at a natural break
    • A chunk stays on one topic
    Two ways to cut the same document.

  • Set a size limit

    Pick a maximum size. This example counts words, and the limit is 8 words per chunk. Real systems often count tokens, because model limits are set in tokens. Smaller chunks give more precise matches, and larger chunks keep more context.

    Size and overlapChunks from 20 wordsWords per chunk
    8 and 238
    4 and 054
    The number of chunks made from the same 20 words, with two settings.
    const words = 20;const size = 8;const overlap = 2;const step = size - overlap;const count = Math.ceil((words - overlap) / step);console.log("Chunks ->", count);

    Output

    Chunks -> 3

  • Overlap keeps a sentence whole

    Repeat the last 2 words of one chunk at the start of the next. Here chunk 2 starts with "days. Refunds", so a sentence at the border stays whole. Overlap costs storage, because each repeated word is stored twice.

    • Chunk 1: Refunds take 5 to 7 business days. Refunds8 words
    • Chunk 2: days. Refunds go to the original card. Contact8 words
    The last two words of chunk 1 repeat at the start of chunk 2.
    const text = "Refunds take 5 to 7 business days. " +  "Refunds go to the original card.";const words = text.split(" ");const size = 8;const overlap = 2;const second = words.slice(size - overlap, size * 2 - overlap);console.log(second.join(" "));

    Output

    days. Refunds go to the original card.

  • Each chunk records its position

    Each chunk stores its document name and position, such as "faq, chunk 3 of 3". The record is used for citations, so an answer can name the exact part of the source. A chunk that starts with "It takes 3 days" is unclear alone, so each chunk should name its topic.

    ChunkPosition recordStarts withWords
    1faq, chunk 1 of 3Refunds take8
    2faq, chunk 2 of 3days. Refunds8
    3faq, chunk 3 of 3card. Contact8
    Three chunks from the example text, with their position records.
    const doc = "faq";const total = 3;const record = doc + ", chunk 3 of " + total;console.log(record);

    Output

    faq, chunk 3 of 3

  • Read sample chunks alone

    Print a few chunks and read each one as if it were the only text. A chunk that ends with "Refunds take" is cut in half, so the size or the overlap needs a change. Check the samples again after each change.

    1. Print chunks
    2. Read each one alone
    3. Cut in half?
    4. Change size or overlap
    The check that catches a bad cut.

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Cutting by characters only. A cut in the middle of a word or a number gives a broken chunk.
  • Using one size for every document. A table and an essay need different sizes, so test each type.
  • Setting the overlap equal to the size. The chunks then repeat the same text and never move forward.

Use it when and skip it when

Is it the right tool?

Use it when

  • Documents are longer than a few paragraphs.
  • Answers must quote a short, exact passage.

Skip it when

  • Each document is already a short record, such as one FAQ entry.
  • The model can read each whole document with room to spare.

Words to know

Terms in this lesson

Chunk
A small part of a document, stored and searched on its own.
Overlap
Words repeated at the end of one chunk and the start of the next.
Size limit
The largest number of words or tokens allowed in one chunk.
Citation
A reference to the exact source of a passage.
Full code