Weekend Challenge · Week 7
Agent vs. Pipeline
A rigorous head-to-head: your agent against your static pipeline, three metrics, no favorites. The interesting result is wherever the agent LOSES.
Your mission
- Run your full eval suite against both systems: quality, latency, token cost
- Break results down by query type: simple factual, multihop, out-of-corpus
- Write a deployment recommendation: which system serves which traffic?
Deliverable
A comparison report with per-query-type tables and your routing recommendation, committed.
Stretch goal
Implement the router you just recommended: simple queries hit the pipeline, hard ones wake the agent. Measure the blended cost and quality.