When Agents Fail: Model or Harness--Who's to Blame?
LLM agents combine models with execution harnesses, so agentic benchmark performance reflects their joint behavior. We argue that harnesses should therefore be evaluated through their interactions with underlying models. We develop a framework combining five complementary measures: (i) Valid Completion Rate (VCR), how...
Shao Zhang, Hang-Fan Zhang, Yuan-Yuan Shi et al.
· 0 citations