AXLE · RAG & Context Engineering
Home / Syllabus / Week 1
Week 1 of 8

When (and When Not) to Retrieve

What language models actually know, where they hallucinate, and the engineering decision that defines every RAG system.

Learning objectives
  • Explain the difference between parametric knowledge and externally grounded knowledge
  • Identify the query categories where retrieval adds value — and where it adds only cost
  • Set up a professional local development environment: Ollama, uv, git
  • Empirically map a model's knowledge boundaries
Study material

Open the full Week 1 study material → Complete lesson: concepts in depth, the full lab with all code, and troubleshooting.

Concepts

1. Parametric knowledge and its limits

Everything a model 'knows' is compressed into its weights during training. That compression is lossy: frequent facts survive; rare ones blur. The model has no flag for 'I never learned this' — which is why it confabulates fluently.

2. The context window as a budget

Whatever the model didn't memorize must arrive through the prompt. The context window is finite, and quality degrades as it fills. Context engineering is the discipline of spending that budget well.

3. The retrieval decision

Retrieval earns its cost on: private data, fresh data, long-tail facts, and auditable citations. It wastes cost on: stable general knowledge, reasoning tasks, and creative work. Production systems route between these cases.

4. Alternatives to RAG

Long-context stuffing, fine-tuning, and tool calls each solve part of the problem. You will be able to argue when each beats retrieval — a question every system design interview and every architecture review asks.

Lab — live session

  1. Install Ollama, pull llama3.1:8b and nomic-embed-text
  2. Create the course repo with uv; first commit
  3. Write hello_llm.py — your first programmatic call to a local model
  4. Probe the model's knowledge boundaries across query categories
  5. Choose your capstone corpus (20+ documents in a domain you can judge)
Checkpoint

Working local environment, committed repo, and a one-page retrieval-decision memo for your capstone domain.

Reading

Lewis et al. (2020) — Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksThe paper that named RAG. Read the intro and §2; skim the rest.

Watch

Andrej Karpathy — [1hr Talk] Intro to Large Language ModelsThe best single hour on what an LLM is and isn't. Watch the sections on the 'LLM OS' and hallucination.

Reflection — bring answers to the next session

  1. Which model failure surprised you most, and why?
  2. If you could fix only one failure category with retrieval, which delivers most value to your capstone users?
  3. When the model didn't know, did it refuse or confabulate? Why does that difference matter in production?
Weekend challenge

Challenge 1: Catch Your Model Lying — Before trusting retrieval to fix hallucination, you need to see hallucination — systematically, in your own model, with your own eyes.

← SyllabusWeek 2 →