VereskLearn

Module 6 · Tools and agents

Lesson 6.7

Many agents and fan-out

Fan-out runs several workers on separate parts of one job at once, then gathers their answers.

Lesson 44 of 622 min read

The problem

Why it matters

Without it

One agent handles every part in turn. A large job takes as long as all its parts added together.

With it

A planner splits the job, the workers run at the same time, and their answers are merged in order.

How it works

  • A planner splits the job

    A planner splits one question into parts that do not depend on each other. The sample has three parts: refund rules, shipping times, and gift card terms. Each part can go to its own worker.

    1. One question
    2. Planner
    3. Refund rules
    4. Shipping times
    5. Gift card terms
    One question, split into three parts.

  • Each worker answers one part

    The askWorker function handles one part and returns its answer. Each answer carries the worker name, so the reader can see which part it came from. The map creates one worker call for each part.

    async function askWorker(name: string, part: string) {  return name + " answered: " + part;}askWorker("worker 1", "refund rules").then(console.log);

    Output

    worker 1 answered: refund rules

  • Promise.all waits for every worker

    Promise.all waits until every worker has finished, then returns the answers in input order. A Promise is a value for a result that is not ready yet. The answers print in the order of the parts, even if a worker finishes later.

    WorkerPartPosition in result
    worker 1refund rules1
    worker 2shipping times2
    worker 3gift card terms3
    Three workers, with the position of each answer.
    const jobs = [  Promise.resolve("refund rules done"),  Promise.resolve("shipping times done"),];Promise.all(jobs).then((answers) => console.log(answers));

    Output

    [ 'refund rules done', 'shipping times done' ]

  • Parallel workers cut the wait, not the cost

    Three workers make three model calls, the same number as one agent working in turn. The sample waits 10 milliseconds per worker, so running together takes about 10 milliseconds, not 30. Rate limits can still stop too many workers at once.

    Waiting time for the three parts, in milliseconds.

Full code

Read the program

See the full code

Common mistakes

What goes wrong

  • Splitting parts that depend on each other. A worker then needs an answer that is not ready yet.
  • Starting too many workers at once. Rate limits and costs grow with each one.
  • Losing the link between an answer and its part. Keep the part name with each answer.

Use it when and skip it when

Is it the right tool?

Use it when

  • The parts are independent and the job is large.
  • Each part can be checked on its own.

Skip it when

  • One question with one source. A single loop is simpler.

Words to know

Terms in this lesson

Worker
One agent that handles one part of a job.
Planner
The step that splits a job into parts.
Fan-out
Starting many workers at once, one for each part of a job.
Promise
A JavaScript value for a result that is not ready yet.
Full code