Prompting, retrieval, and fine-tuning solve different failure classes

Prompting changes inference context, retrieval supplies external evidence, and fine-tuning changes model parameters or behavior. None guarantees correctness, freshness, safety, or deterministic output, and combinations introduce separate data and lifecycle risks.

Mental model

• RAG supplies retrieved non-parametric memory at inference time in addition to model parameters. Use retrieval for evidence with provenance, not as a factuality guarantee.
• Instruction fine-tuning with human feedback changed behavior in the reported model and prompt distribution, while errors remained. Do not promise general alignment or correctness from fine-tuning.
• OpenAI's model optimization workflow couples evals, prompting, and fine-tuning. Adopt an eval-driven sequence instead of a fixed vendor-independent recipe.
• Fine-tuning jobs produce new model artifacts from explicit training data and configuration. Treat tuned models as versioned deployable artifacts with rollback.

Guarantees and boundaries

• Each approach changes a different part of the system and has a separate version.
• A held-out test set is not training data.
• Fresh retrieved evidence does not force the generator to use it correctly.

Diagram scenarios

Prompting baseline first. A template and examples are the lowest-lifecycle-cost intervention, but still require regression evaluation.

Retrieval for changing evidence. RAG can update attributable knowledge without changing weights, while adding retrieval failures and injection risk.

Fine-tuning as a model lifecycle. Training requires curated data, held-out evaluation, provenance, release gates, and rollback.

Combination requires ablation. Prompt, retrieval, and tuned weights can mask each other's regressions and compound cost.

Architecture decision

Start with a versioned baseline and task evaluation. Use prompting for interface and examples, retrieval for attributable changing knowledge, and fine-tuning only when a curated dataset and stable behavioral objective justify training operations. Combine methods only when an ablation proves each component's value.

Failure modes

• Fine-tuning can memorize sensitive or erroneous examples.
• Retrieval can return stale, unauthorized, or injected passages.
• Prompt changes can regress previously passing cases without an API error.

Operational checklist

• Name the observed failure before selecting a technique.
• Keep data provenance, consent, licensing, deletion, and contamination controls.
• Run ablations under the same model, corpus, prompt, and eval versions.
• Budget training, indexing, serving, review, rollback, and monitoring—not only token charges.

Related concept

Primary sources

1. https://arxiv.org/abs/2005.11401
2. https://arxiv.org/abs/2203.02155
3. https://platform.openai.com/docs/guides/model-optimization
4. https://platform.openai.com/docs/guides/fine-tuning