VereskBuild an AI assistant, one piece at a time
By the end you can build an assistant that answers questions from your own documents. It finds the right passage, writes an answer, and names its sources.
You will also test it, so you can tell when a change makes it worse. 11 modules, 62 lessons. 0 visited.
Module 0
Start here
What the course builds, and how each lesson is laid out.
Module 1
Talking to a model
How a model takes text in and gives text back, and how text becomes numbers.
- 1.1One model call
- 1.2Tokens and cost
- 1.3Temperature and why answers vary
- 1.4Embeddings: text as numbers
- 1.5Cosine similarity
Module 2
Measure first: evals
How to test an assistant against fixed questions, so changes can be judged.
- 2.1The golden set
- 2.2Retrieval metrics
- 2.3Checking answers and LLM-as-judge
- 2.4Repeat runs and flaky results
- 2.5Baselines, diffs, and regressions
- 2.6Severity buckets and control cases
Module 3
Preparing documents
How raw files become small, clean, searchable parts.
Module 4
Retrieval
How the assistant finds the passages that answer a question.
- 4.1Vector search
- 4.2Metadata filters
- 4.3Keyword search
- 4.4Hybrid search and RRF
- 4.5Reranking
- 4.6Query rewriting
- 4.7Parent-child and compression
- 4.8Corrective retrieval
- 4.9Text to SQL
- 4.10Graph retrieval
- 4.11Multimodal retrieval
- 4.12What retrieval cannot fix
Module 5
Writing the answer
How retrieved passages become a checked, cited answer.
- 5.1Building the context
- 5.2The prompt
- 5.3Structured output with a schema
- 5.4Coherence rules
- 5.5Citations, refusing, and escalating
- 5.6Long context and attention
Module 6
Tools and agents
How a model calls functions in a loop, and how to organize many of them.
- 6.1What a tool is
- 6.2The tool-calling loop
- 6.3Lookup tools and search tools
- 6.4Wrapping a messy API
- 6.5Routing questions
- 6.6Frameworks or a hand-written loop
- 6.7Many agents and fan-out
Module 7
MCP
A standard way for a model host to reach tools and data over a protocol.
- 7.1Host, client, server, and JSON-RPC
- 7.2A stdio server with tools
- 7.3Resources and prompts
- 7.4HTTP transport and tokens
- 7.5Writing a client
Module 8
Safety
How to keep untrusted text and risky actions from doing harm.
- 8.1Prompt injection
- 8.2Guardrails and confirm before write
- 8.3Credentials
- 8.4What data leaves your servers
Module 9
Running it for real
What changes when real users arrive: streaming, logs, cost, caching, drift, and debugging.
- 9.1Streaming answers with citations
- 9.2Tracing and logs
- 9.3Tracking cost
- 9.4Caching
- 9.5Drift
- 9.6Debugging a bad answer
Module 10
Beyond retrieval
When to change the model itself, and how to choose between the options.