My AI judges agreed with themselves. Not with each other.
Each judge mostly agreed with itself. On borderline cases, they often didn't agree with each other, and that difference alone moved a pass rate by 27 points.
Topic
Evaluating LLM outputs
Judges, tests and ground truth: how to tell whether a model’s answer is right, and how sure you can be.
I swapped the AI model and every test still passed. That was the problem.
Unit tests protect the code around a model. They can't tell you whether a new model still gets the answers right, and that's the change most likely to break things.