Evaluating LLM Apps

How do you grade an essay by machine? That is the problem of evaluating language model apps. There is no single right answer to compare against. Still, you can build a reliable evaluation, with a mix of simple checks, and careful judging. Let's see the options, from cheapest to richest.

• No single right answer
• Mix cheap checks and judging
• Build a golden set

A ladder of evaluation methods

There is a ladder of methods. At the bottom, simple rules. Does the answer contain the needed fact? Is it short enough? Does it avoid forbidden words? These are fast, cheap, and exact. Next, reference comparison, how similar is the answer to a known good one. Then, model graded evaluation, where another language model scores the answer, against a rubric, like correct, grounded, polite. This scales well, but must be checked against human judgement. And at the top, human review, slow and expensive, but the final authority. Use the cheap methods on everything, and the expensive ones on samples.

Building a golden set

Everything rests on a good golden set, a collection of realistic questions, with the behaviour you expect. Start with real user questions, from logs, with private data removed. Include easy cases, hard cases, and traps, like questions the model should refuse, or answers that are not in your documents. Add every bug you ever fix as a new test, so it can never return. Keep the set versioned, and grow it steadily. Even fifty good cases are far better than none.

The pitfalls of LLM judges

Model graded evaluation is powerful, but has pitfalls. Judges tend to prefer longer answers. They may favour answers written in their own style. They can be inconsistent, so run them several times, or use low randomness. And they can share the same blind spots as the model being judged. To stay honest, write a clear rubric, calibrate the judge against a few hundred human labels, and measure how often they agree. Treat the judge as a measuring instrument, that itself needs testing.

Summary