System Design Cases
AI Evals
AI evals concept page: offline eval suite (golden dataset, F1, LLM-as-judge), RAG eval pipeline (recall@k/MRR + faithfulness/answer-relevance via Ragas/DeepEval/TruLens), production A/B test with implicit signals and feedback loop into goldens.
AI evaluations: versioned evidence, slices, and known blind spots
An evaluation estimates behavior on defined cases, rubrics, graders, and sampling conditions. One benchmark, average, or judge cannot cover all capabilities, risks, languages, or deployment shifts.
Mental model
- HELM argues for broad scenario coverage, multiple metrics, transparency, and explicit recognition of evaluation incompleteness. Use multidimensional reports and state what remains untested.
- NIST's Generative AI Profile recommends governance, pre-deployment testing, measurement, and incident disclosure actions. Connect evals to lifecycle risk management rather than leaderboard theater.
- OpenAI eval tooling defines datasets, runs, and graders as versioned evaluation components. Pin configs and preserve per-case evidence.
- LLM judges have measured position, verbosity, and self-enhancement biases. Calibrate judges and never use one unvalidated grader as ground truth.
Guarantees and boundaries
- A report identifies the exact system, cases, grader versions, and failed examples.
- Release gates are declared before seeing candidate results.
- Evaluation scope and known exclusions are explicit.
Diagram scenarios
Pre-deployment evaluation. One run pins the system, dataset, prompts, graders, and environment.
Average pass, slice fail. An aggregate score can hide a severe regression in a language, cohort, task, or failure mode.
Contamination and eval overfitting. Repeated exposure to public or internal test cases can make a score stop estimating unseen behavior.
Production failure becomes a governed case. Online incidents inform future tests without leaking private data or training directly on every complaint.
Architecture decision
Start from product requirements and harms, maintain private and adversarial cases with provenance, use deterministic checks where possible, calibrate model graders against human labels, report slices and uncertainty, and block releases only on predeclared gates. Keep online monitoring separate from reusable offline evidence.
Failure modes
- Benchmark contamination and repeated tuning can inflate scores.
- A grader can share blind spots or be manipulated by candidate text.
- Online traffic can differ from every offline dataset.
Operational checklist
- Test the full application path including retrieval, tools, policies, and fallbacks.
- Keep private holdouts and rotate adversarial cases.
- Report slices, denominators, uncertainty, cost, latency, safety, and abstention.
- Sample human review and measure grader agreement and drift.