What Makes Software Issue Resolution Tasks Difficult for Agents?
Background. Advances in agentic systems are simultaneously, and rapidly, saturating benchmarks. Despite this often discussed phenomena, benchmark scores remain difficult to interpret due to the lack of control and characterization of task difficulty. More specifically, we currently have little understanding of what mak...
Ebtesam Al-Haque, Brittany Johnson
· 0 citations