LLM-as-judge

LLM-as-judge uses a (usually frontier) model to score the outputs of your production model against a rubric you provide. It is the standard pattern for evaluating open-ended generation tasks where deterministic scoring is impossible. Calibrate the judge against human ratings on a sample before trusting its scores at scale, and watch for known biases (judges prefer longer outputs, prefer first-listed options in pairwise comparisons).