Evaluation lab · San Francisco · London · Zürich
Reinforced Labs Request a briefing
Glossary

LLM-as-a-judge

Also called: LLM judge, model-based grader, AI judge, autograder, grader calibration

LLM-as-a-judge uses a language model to grade outputs against a rubric. It is trustworthy only per category and after calibration against human labels.

LLM-as-a-judge is the practice of using a language model to score or compare other models' outputs against a written rubric. It makes evaluation of open-ended text cheap and fast, but a judge is a measuring instrument with its own error rates, and those must be measured against human labels before its scores mean anything.

Common formats

  • Pointwise: the judge assigns a label or score to one response (pass/fail, 1 to 5).
  • Pairwise: the judge picks the better of two responses, which is easier to do consistently and maps onto preference data.
  • Reference-guided: the judge compares a response with a reference answer or source documents, as in faithfulness checks.

Grader calibration

Calibrating a judge means scoring a gold set of expert-labelled items and measuring agreement per category: sensitivity (true failures caught), specificity (passes correctly passed) and Cohen's kappa. If those rates are known, the observed failure rate can be corrected to an estimate of the true rate :

This is the Rogan-Gladen correction from diagnostic testing. It assumes the judge's error rates on the gold set carry over to the data being scored.

Worked example

Illustrative numbers: a judge flags 20% of responses as unsafe. On a gold set it has sensitivity 0.90 and specificity 0.95. The corrected estimate is . The raw number overstates the failure rate by about two points because 5% of safe responses are wrongly flagged.

Known biases

  • Position bias: in pairwise mode, preferring the first or second answer. Mitigate by scoring both orders.
  • Verbosity bias: preferring longer answers regardless of quality.
  • Self-preference: favouring outputs that resemble the judge's own style.
  • Rubric drift: small prompt changes shifting scores. Version the judge prompt and model like code.

Why it matters

Most modern eval suites and many RLAIF pipelines depend on model graders. An uncalibrated judge can make a regression invisible or invent one, and a judge used as a reward is exposed to reward hacking. Calibrate per category, recheck after any change to the judge, and keep humans on the categories where agreement stays low.

Sources

  1. Lianmin Zheng et al.. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, 2023. arxiv.org

Know the number before you ship.

A 45-minute working session with an evaluation engineer. Bring one use case; leave with a threat model, a draft eval plan with sample sizes, and an honest read on where you stand.

Request a briefing →Read our methods