Evaluating mainstream coding agents on DEPBENCH, a benchmark consisting of 203 real-world dependency-upgrade tasks across five package ecosystems spanning five language communities, each involving hidden code-level changes that require source code adaptation.
Zi-Jian Luo, Runzhi He, Peng-Fei Gao et al.· 0 citations
A systematic study of observability in agent-generated systems reveals a consistent gap between diagnostic semantics at the source level and fault signals at runtime, suggesting that current evaluations focusing primarily on functional correctness may overlook observability as an important dimension of practical softwa...
This paper discusses how CloudWeaver scopes the context of individual agent sessions with local views of cloud resources and coordinates concurrent management operations on shared cloud resources, and offers strong safety guarantees and attributable feedback in the presence of conflicting intents, while preserving conc...
Minghao Li, Ziqian Liu, Ziyu Mao et al.· arXiv.org· 0 citations
Coding agent infrastructure is shifting from harness engineering toward loop engineering as coding agents are deployed for sustained long-horizon software development. Existing benchmarks often center on localized tasks or end-state outcomes, offering limited insight into sustained execution. We introduce LOOPSBENCH, a...
This work presents Change2Task, a system grounded in repository history that converts merged pull requests into verified tasks on healthy modern revisions of the same repository, and provides executable data for coding agent training and evaluation while reducing repeated environment setup, storage, and task constructi...