Adaptive Graph Harness Optimization for LLM Agents
Writes a coding agent’s harness as a graph and finds that an enforced test check with feedback and bounded retries raises bug-repair rates by 14 to 24 points when tests are visible, while no graph helps when they are hidden. On 232 held-out tasks the check was confirmed (+6.5 points over the plain loop), and the graph returned by a search over 3,600 graphs showed no demonstrated additional benefit (+0.6 points, 95% interval −1.3 to 2.8); a follow-up on 900 fresh tasks gave the same answer (+0.2 points, −1.3 to 1.7), with the check 24 points ahead of the plain loop. For the case with no tests, a premise test asked whether the model could write its own check: its checks rejected 29% of known wrong patches but also 19% of bug-free repositories against a 2% limit, so that approach was stopped before a repair experiment.
