Agents fail in the middle, not at the end. A wrong tool argument at step three turns into a confident, wrong answer at step nine, and a final-answer eval only sees the last part.

Score every step

Holdout records each tool call as its own span with the input, the arguments and the result. Judges can score a single step, so a regression shows up where it starts.

Replay the run

When a case fails, open the trace and replay it against main. The diff shows the first step where the two runs part ways, which is usually the fix.