VereskLearn

Module 5 · Writing the answer

Lesson 5.4

Coherence rules

Coherence rules check that a valid reply also makes sense against the sources.

Lesson 35 of 622 min read

The problem

Why it matters

Without it

A reply with the right shape can still cite a source that was never found, or hide a conflict.

With it

Each reply passes the shape check and the coherence rules. A reply that fails is sent back, not shown.

How it works

  • Shape and meaning are checked apart

    Shape checks the type of each field. Coherence checks the meaning, by comparing the reply with the sources that search returned. A reply can pass the shape check and still fail coherence.

    Shape

    • Each field has the right type
    • Checked with typeof

    Coherence

    • Each cited id is a real source
    • A conflict is escalated
    The two layers of checks on one reply.

  • Each cited id must be a known source

    knownIds holds the ids that search returned for this question. Each id in sourceIds must be in that set. The id manual-99 is not in the set, so the reply is reported.

    • faq-12known
    • manual-99unknown, reported
    Cited ids, checked against the ids that search returned.
    const knownIds = new Set(["faq-12", "faq-40"]);for (const id of ["faq-12", "manual-99"]) {  const ok = knownIds.has(id);  console.log(id + (ok ? " known" : " unknown"));}

    Output

    faq-12 known
manual-99 unknown

  • A conflict needs escalation

    A non-empty conflicts list means two sources disagree. In that case escalate must be true, so a person makes the decision. The broken reply has a conflict and escalate set to false, so it is reported.

    ReplyConflictsEscalateResult
    Good0falsepasses
    Broken1falsereported
    Escalated1truepasses
    Three replies checked against the conflict rule.
    const conflicts = ["faq-12 says 5 to 7 days"];const escalate = false;if (conflicts.length > 0 && !escalate) {  console.log("conflict without escalation");}

    Output

    conflict without escalation

  • Every error is collected

    The function gathers every error into one list instead of stopping at the first. A good reply gives an empty list. The broken reply gives two errors, so one check shows every problem at once.

    Errors found for each reply.

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Treating a passed shape check as proof. A valid shape can still carry a wrong answer.
  • Testing only good replies. Also test broken replies, so you know each rule fires.
  • Returning a single yes or no. A list of named errors tells the next step what to fix.

Use it when and skip it when

Is it the right tool?

Use it when

  • The reply must match the sources, not only the shape.
  • A wrong answer would cost more than a rejected reply.

Skip it when

  • The reply is free text with no sources or fields to check.

Words to know

Terms in this lesson

Coherence
Whether a reply makes sense with the facts that were retrieved.
Conflict
Two sources that give different answers to the same question.
Escalate
Hand the case to a person instead of answering it.
Set
A container that stores each value once and answers membership questions fast.
Full code