Automated harness optimization can substantially improve LLM agents by iteratively updating their prompts, tool interfaces, and control logic from execution feedback. However, existing methods primarily optimize how the harness is updated while largely fixing which training scenarios generate the feedback that drives t...
Sungho Park, Wonjoong Kim, Jue Zhang et al.· 0 citations
A systematic study of observability in agent-generated systems reveals a consistent gap between diagnostic semantics at the source level and fault signals at runtime, suggesting that current evaluations focusing primarily on functional correctness may overlook observability as an important dimension of practical softwa...
The proposed AutoSaddler, an automatic harness optimization framework that formulates harness improvement as an offline learning problem and iteratively updates the harness using failure signals from mini-batches, suggests that automatic harness optimization is a promising path toward more performant and reliable agent...
Sungho Park, Wonjoong Kim, Rongyuan Tan et al.· 15 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.