Prompt engineering: versioned interface plus evaluation

A prompt is application code that shapes a probabilistic model call. Examples, instructions, context ordering, model version, decoding, and tool schemas interact, so a prompt that works on anecdotes is not a reliable contract.

Mental model

• Large language models exhibit in-context task behavior under specific model and benchmark conditions. Describe demonstrations as an empirical technique rather than training or a guarantee.
• Chain-of-thought demonstrations improved selected reasoning benchmarks in the reported experiments. Do not claim hidden reasoning, correctness, or universal gains from asking for steps.
• OpenAI recommends model-specific prompting and evaluation because model behavior can change across versions. Pin model snapshots and test prompts as code.
• Prompt injection and improper output handling remain application security risks. Keep permissions and output validation outside the prompt.

Guarantees and boundaries

• Schema validity is necessary but not sufficient for semantic correctness or authorization.
• A pinned template identifies what was requested; it does not make model output deterministic.
• Tool permissions derive from application identity and policy, never generated text.

Diagram scenarios

Versioned prompt call. The request is assembled from typed fields and a pinned template-model pair.

Examples are tested, not assumed. Few-shot and reasoning prompts show empirical gains on some tasks and models, not universal improvement.

Prompt injection does not grant authority. Natural-language separation can reduce mistakes but cannot enforce permissions.

Malformed or unsafe output recovery. A schema retry is bounded and cannot turn a risky action into success.

Architecture decision

Version prompts with their model snapshot and input schema, evaluate representative and adversarial cases before release, validate structured outputs, and keep authorization outside natural-language instructions. Treat user and retrieved text as untrusted data and bound retries, tokens, latency, and spend.

Failure modes

• Prompt or model changes can create silent regressions.
• User, retrieved, image, or tool text can inject competing instructions.
• Boundless repair retries can multiply cost and side effects.

Operational checklist

• Version prompt, model, decoding, tool schema, and evaluation set.
• Test typical, tail, multilingual, adversarial, and malformed inputs.
• Validate syntax, references, permissions, and business invariants.
• Record request IDs and versions without logging unnecessary sensitive text.

Related concept

Primary sources

1. https://arxiv.org/abs/2005.14165
2. https://arxiv.org/abs/2201.11903
3. https://platform.openai.com/docs/guides/prompt-engineering
4. https://genai.owasp.org/llm-top-10/