AI coding agents now write much of the code that ships, yet whether that code is secure remains unclear. We introduce the Last Secure Code Benchmark, a comprehensive evaluation of secure code generation by coding agents. Built from 150 real vulnerabilities, its 450 tasks span seven widely used languages (Java, C, C++,...
Tian-Yi Wu, Chao Ge, Peng Chen et al.· 0 citations
Harbor Adapters are developed that port more than 80 benchmarks to evaluate arbitrary agents, and validated through rigorous code review and parity experiments, and Harbor-Index is introduced, a curated set of 82 difficult, diverse, and high-quality tasks spanning 29 benchmarks to support more reliable and comprehensiv...
Lin Shi, Hao-Wei Lin, Zi-Xuan Zhu et al.· 1 citation
ACToR identifies critical tokens during generation and triggers targeted retrieval on demand to provide repository context at these decisive positions, and designs a position-aware weighting method for dense retrievers to prioritize context that is more informative for generation.
Ke-Feng Duan, De-Wu Zheng, Yan-Lin Wang et al.· 0 citations
SWE-Bench ProMax is introduced, an expert-curated, multilingual code refactoring benchmark of 170 instances drawn from real commits across seven programming languages, which presents a meaningful and unsaturated challenge for current AI coding agents.
Yu-Ling Shi, Jing-Heng Xu, Kelin Fu et al.· 6 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.