Production: Secure, Observable, Deployable

The final mile: your system meets adversaries, latency budgets, and real users. Capstone week.

Concepts

1. The RAG attack surface

Your index ingests documents; documents can contain instructions. Indirect prompt injection — attacks hiding in the corpus itself — is ranked the #1 LLM vulnerability by OWASP. Defense is layered: input sanitation, privilege separation, output validation.

2. Access control on the index

If users have different permissions, retrieval must respect them — filtering at query time, not after generation. A RAG system that leaks one confidential chunk has failed entirely.

3. Observability

Log every retrieval: query, candidates, scores, what made the context window. When answers degrade in production, these traces are the only way to run your Week 5 diagnosis on live traffic. Add drift detection: corpora change, embeddings go stale.

4. Performance engineering

Semantic caching, streaming tokens, async retrieval, and index update pipelines. Latency budgets are design constraints, not afterthoughts.

Lab — live session

1. Red-team your own system: plant an injection in your corpus, watch it fire, then defend
2. Add structured tracing and retrieval logs
3. Wrap the system in FastAPI with streaming; add a minimal chat UI
4. Containerize and load-test; document your latency and cost profile

Reading

Watch

Reflection — bring answers to the next session

1. Did your injection defense survive a second, sneakier attempt?
2. Which trace fields actually helped you debug — and which were noise?
3. Explain your system to a non-technical stakeholder in three sentences. Did they get it?

Final exercise set — capstone review

Ten exercises spanning the whole program. Complete them before your capstone defense; several make excellent warm-up material for the presentation itself.

1. The retrieval case (Week 1). In three sentences a stakeholder would understand, justify why your capstone needs retrieval at all — and name one query type it should answer without retrieving.
2. BM25 by hand (Week 2). Given a 3-document toy corpus, hand-compute the BM25 score of one query against each document (k1=1.5, b=0.75). Verify with rank_bm25.
3. Metrics by hand (Week 2). From a printed result table of 5 queries, compute Recall@5 and MRR on paper before checking with code.
4. The prediction game (Week 3). Write 5 fresh queries for your corpus and predict, before running anything, whether BM25 or embeddings wins each. Score your predictions.
5. RRF on paper (Week 4). Given two ranked lists of 5 documents, compute the fused RRF ranking by hand (k=60). Explain in one sentence why no score normalization was needed.
6. Failure triage (Week 5). For a wrong answer transcript, write the exact decision procedure you'd follow to classify it as retrieval, generation, or chunking failure — then apply it.
7. Citation scaffold (Week 6). Draft the prompt block that enforces your citation format, and demonstrate on 3 questions that citations point at genuinely supporting text.
8. Trace the hops (Week 7). Author one new multihop question and hand-trace the retrieval hops your agent takes, comparing against what it should have done.
9. One novel attack (Week 8). Write one injection payload not used in your challenge, predict which defense layer stops it, then test the prediction.
10. The budget memo (Week 8). Write your capstone's latency and cost budget per query, identify the single biggest lever for each, and state what you'd sacrifice first under load.