LLM-as-judge: calibrated measurement with adversarial controls

A model judge is another probabilistic model call. It can scale rubric application, but agreement depends on task, prompt, reference, candidate order, verbosity, model family, and adversarial content.

Mental model

• MT-Bench research reports both usefulness and position, verbosity, self-enhancement, and reasoning limitations of LLM judges. Make bias tests and human calibration mandatory.
• G-Eval reports task-specific human correlation and notes bias toward LLM-generated text. Do not generalize one summarization experiment to every rubric.
• HELM emphasizes multi-metric, scenario-specific, transparent evaluation. Use model judges as one signal within a broader evaluation.
• OpenAI graders support multiple deterministic and model-based grader types. Prefer objective checks where objective truth is available.

Guarantees and boundaries

• Candidate IDs and order randomization are recorded.
• Judge outputs must match a strict schema and known candidate set.
• Agreement and disagreement are reported against a human-labeled calibration sample.

Diagram scenarios

Blinded pairwise judgment. Candidate identities are hidden and order is randomized before a schema-constrained judgment.

Order and style bias test. Swapping order and controlling length exposes judge instability.

Candidate prompt injection. A candidate can ask the judge to ignore the rubric, award itself a high score, or reveal secrets.

Independent judges disagree. More judges do not create truth; disagreement is evidence about uncertainty and rubric ambiguity.

Architecture decision

Use deterministic graders for objective properties, blind and randomize pairwise comparisons, validate judge outputs, calibrate on a representative human-labeled set, and report disagreement. Never let candidate text become judge instructions or let a judge alone authorize a high-impact release.

Failure modes

• Position, verbosity, style, self-preference, and shared-model bias can change verdicts.
• Candidate text can prompt-inject the judge.
• Judge or prompt updates can drift scores even with identical candidates.

Operational checklist

• Blind identities and randomize or swap order.
• Pin judge, prompt, rubric, decoding, and candidate formatting.
• Measure human agreement by slice and retain disagreements.
• Never use generated confidence as calibrated probability without evidence.

Related concept

Primary sources

1. https://arxiv.org/abs/2306.05685
2. https://arxiv.org/abs/2303.16634
3. https://arxiv.org/abs/2211.09110
4. https://platform.openai.com/docs/guides/graders