The method behind 94% agreement: sampling, disagreement review and when to keep a human in the loop.

What we measured

We ran every variant against the same 240 production cases, scored each with a calibrated judge, and compared pass rate, cost per 1k calls and p95 latency against main.

What we learned

Most regressions came from instructions that looked redundant. Keep the ones that constrain format, citations and refusals, and test every removal on real cases before it ships.