Models as Judges

Is this reply kind? Is it accurate? Does it solve the problem? Code can't answer that. But a model can, if you give it a clear rubric. It's like having a very fast, slightly unreliable teaching assistant.

• Is this reply kind?
• Code can't answer that
• A fast teaching assistant

A rubric-based judge

The rubric says what a five looks like, and what a one looks like. We give the judge the facts, and a reply. One reply is honest. The other one promises something that nobody can guarantee. Run it. The honest reply scores high. The false promise scores low, and the judge says why. Run this across your whole golden set, and you've got a quality score.

from pydantic import BaseModel
from llm import ask

class Grade(BaseModel):
    score: int
    reason: str

RUBRIC = """Score 1 to 5. 5 = solves the problem, kind, accurate to the facts.
1 = wrong, rude, or invents facts."""

facts = "Order B42 shipped two days ago. Delivery takes four days."
for reply in ["B42 shipped two days ago, so it should arrive in about two days.",
              "Your order arrives tomorrow, guaranteed! Sorry about the delay."]:
    g = ask(f"{RUBRIC}\nFacts: {facts}\nReply: {reply}", schema=Grade)
    print(f"{g.score}/5 | {reply[:38]} | {g.reason[:42]}")

5/5 | B42 shipped two days ago, so it should | The response is accurate, directly answers
1/5 | Your order arrives tomorrow, guarantee | The reply provides inaccurate information

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

Checking groundedness

For agents that use documents, the key question is, did it make this up? This check compares an answer against its source, and lists any claims that aren't supported. Run it. The first answer is fine. The second invents a lifetime warranty, and it gets caught. This is one of the most valuable checks that you can run in production.

from pydantic import BaseModel
from llm import ask

class Check(BaseModel):
    grounded: bool
    unsupported: list[str]

source = "Wheelsets have a two-year warranty against manufacturing faults."

for answer in ["Your wheelset has a two-year warranty against manufacturing faults.",
               "Your wheelset has a lifetime warranty, including crash damage."]:
    c = ask(f"Source: {source}\nAnswer: {answer}\n"
            "Is each claim supported by the source?", schema=Check)
    print(c.grounded, c.unsupported)

True []
False ['Your wheelset has a lifetime warranty', 'including crash damage']

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

Trusting your judges

Judges need testing too. Compare their scores with human ratings on some examples. Use simple scores, such as pass or fail, as they're more reliable than fine scales. Ask for the reason before the score. Use one question per judge. And have people review a sample each week, so that you can see any drift.

Summary