VereskLearn

Module 0 · Start here

Lesson 0.1

What you will build

You will build an assistant that answers questions from documents and names the source of each answer.

Lesson 1 of 622 min read

The problem

Why it matters

Without it

A person searches a long manual by hand. Finding one answer can take many minutes.

With it

The assistant finds the right passage, writes an answer, and names the source. The reader checks the source in seconds.

How it works

  • The five parts of the assistant

    Five parts work together: the documents, the chunks, the find part, the model, and the answer with its sources. The sample program in this lesson does not call a model. It shows only the shape of the find part.

    1. Documents
    2. Chunks
    3. Find
    4. Model
    5. Answer with sources
    The five parts, left to right. Each answer comes from the documents, not from memory alone.

  • Chunks: small parts of a document

    A 2,000-word manual becomes 10 chunks of about 200 words each. Each chunk is stored on its own, so the find part can return only the relevant part. The sample program uses three short FAQ entries instead.

    • Chunk 1200 words
    • Chunk 2200 words
    • Chunk 3200 words
    • Chunk 4200 words
    • Chunk 5200 words
    • Chunk 6200 words
    • Chunk 7200 words
    • Chunk 8200 words
    • Chunk 9200 words
    • Chunk 10200 words
    One 2,000-word manual, cut into 10 chunks of about 200 words each.

  • Keyword count scores each FAQ entry

    The find part splits the question and each FAQ question into lowercase words. It counts the shared words, and that count is the score. "My password does not work" shares "my" and "password" with the reset entry. That entry scores 2, so its answer is returned. Later lessons replace this word count with a better measure of meaning.

    • How do I reset my passwordscore 2, best match
    • When will my order shipscore 1
    • How do I get a refundscore 0
    The three FAQ entries from the sample program, ranked for "My password does not work".
    // The question and one FAQ question, split into words.const asked = new Set("my password does not work".split(" "));const faq = "how do i reset my password".split(" ");const shared = faq.filter((w) => asked.has(w));console.log("Shared ->", shared.join(", "));console.log("Score ->", shared.length);

    Output

    Shared -> my, password
Score -> 2

  • The model writes from the matching chunks

    The model gets the question and the matching chunks. It is told to answer only from those chunks and to name the source of each statement. Without the source, a reader cannot check the answer.

    Answer alone

    • Refunds take 5 to 7 days.

    Answer with source

    • Refunds take 5 to 7 days.
    • Source: refund guide, section 2
    The same answer, with and without a named source.

  • Checks with a fixed list of questions

    Run the assistant on the same 20 questions after each change. The count of correct answers is the score. Fewer correct answers than before means the change made things worse.

    Correct answers out of 20 fixed questions, after each of three changes.

  • A log of real use

    Each request is logged with its question, its sources, and its cost. A request with no source, like the gift card question, shows a gap in the documents.

    TimeQuestionSourcesCost
    09:14Can I get a refund?1$0.002
    09:20Gift cards for kids0$0.002
    09:25How do I reset my password?1$0.002
    Three logged requests. The highlighted one found no source, so the documents do not cover it.

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Expecting word counting to understand meaning. It only counts shared words.
  • Skipping the check step. Without tests, a change can make answers worse and nobody notices.
  • Showing an answer without its source. A reader cannot verify an answer that has no source.

Use it when and skip it when

Is it the right tool?

Use it when

  • You want answers from your own documents, with sources.
  • You need to explain how a document assistant is built.

Skip it when

  • You only need a fixed list of answers. A plain list is simpler.
  • You need one exact record, such as an order status. Use a direct lookup.

Words to know

Terms in this lesson

Chunk
A small part of a document, stored on its own.
Model
A program that reads text and writes text back.
Tool
A function the assistant can call, such as a record lookup.
Eval
A test that checks answers against a fixed list of questions.
Full code