RAG: authorized evidence retrieval with explicit uncertainty

Retrieval-augmented generation combines a retrieval system with a generative model. Retrieval can miss, return stale or unauthorized content, or surface malicious instructions; generation can ignore or misstate evidence.

Mental model

• RAG combines parametric generation with retrieved non-parametric memory and was evaluated on knowledge-intensive tasks. Teach the architecture without converting paper results into universal factuality guarantees.
• Dense Passage Retrieval evaluates dual-encoder retrieval against specific open-domain QA datasets. Treat dense retrieval quality as workload-specific and preserve lexical fallback where exact terms matter.
• Long contexts do not guarantee uniform use of evidence across positions. Bound and order context based on evals; more passages can hurt.
• Prompt injection, sensitive disclosure, vector weaknesses, misinformation, and unbounded consumption are recognized LLM application risks. Treat retrieval content as untrusted and apply least privilege plus bounded budgets.

Guarantees and boundaries

• Authorization is evaluated from current policy, not inferred from vector proximity.
• Citations preserve source and version; they do not by themselves prove that every claim is supported.
• A no-answer outcome is allowed and observable.

Diagram scenarios

Grounded answer with provenance. The system retrieves authorized evidence, generates a candidate, and verifies citation support before release.

Insufficient evidence is a valid result. Retrieval can miss or conflict; the generator must not invent a conclusive answer.

Retrieved prompt injection. A document can tell the model to ignore policy or exfiltrate data; retrieved text remains untrusted.

Stale or revoked evidence. Index visibility and source authorization can change between ingestion and answer generation.

Architecture decision

Apply identity and authorization before retrieval and again to returned evidence. Use hybrid retrieval and reranking only when measured on a versioned eval set. Treat retrieved passages as untrusted data, require provenance, allow an explicit insufficient-evidence result, and never make a high-impact action solely from generated text.

Failure modes

• Retrieval miss, stale index, or filter starvation can look like model ignorance.
• Poisoned passages can attempt indirect prompt injection.
• A timeout after a model call is an uncertain operational outcome, not evidence of no billing or no generation.

Operational checklist

• Version corpus, chunker, embedder, index, reranker, prompt, and generator.
• Evaluate retrieval, attribution, answer quality, safety, latency, and cost separately.
• Red-team direct and indirect injection plus cross-tenant retrieval.
• Keep high-impact actions behind deterministic authorization and confirmation.

Related concept

Primary sources

1. https://arxiv.org/abs/2005.11401
2. https://arxiv.org/abs/2004.04906
3. https://arxiv.org/abs/2307.03172
4. https://genai.owasp.org/llm-top-10/