Coding agents are increasingly asked to build software rather than patch it, yet benchmarks for from-scratch repository construction are mostly limited to a single language and depend on manually curated tasks. We introduce Zero2Repo, a benchmark in which an agent receives a product requirements document, an interface...
Pei Yang, Tian-Yu Shi, Yu-Hang Yao et al.· 0 citations
This workshop aims to define a roadmap for a world where AI Teammates and human developers build the future together, anchored by the launch of the AIDev dataset, which provides the empirical evidence needed to understand the behaviors of AI Teammates.
Hao Li, Haoxiang Zhang, Jie M. Zhang et al.· Proceedings of the 32nd ACM...· 2 citations
We are witnessing the emergence of Agentic Software Engineering (SE~3.0), where AI agents act as autonomous AI Teammates performing complex tasks such as coding, debugging, and testing. As AI Teammates generate a vast new category of digital artifacts, they introduce unique opportunities and challenges related to human...
Hao Li, Haoxiang Zhang, Jie M. Zhang et al.· Proceedings of the 32nd ACM...· 0 citations
An empirical study of Claude Code plugin marketplaces finds that the marketplace is expanding rapidly, plugin-touching commit activity growing 8.8x over six months after the October 2025 launch, and plugins targeting Software Engineering tasks accounting for 61.3% of all plugins.
Ahmed Hereiz, Yingzhe Lyu, Hao Li et al.· 0 citations
The first controlled longitudinal study that isolates the scaffolding’s contribution to agent quality over time is conducted, revealing that despite continuous development activity and growing codebase complexity of the scaffoldings, there is no statistically significant improvement in SWE-bench benchmark score across...
This paper empirically study the development and release evolution of five major open-source agent harnesses, revealing extreme release velocities exceeding two releases per day and thousands of issues within months, and performs the first controlled longitudinal study that isolates the agent harness contribution.
O. Sghaier, Hao Li, Bram Adams et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.