Modern coding agents can deliver increasingly large repository-level changes, and recent benchmarks reflect this by emphasizing long-horizon tasks with large reference implementations. Many benchmarks evaluate coding agents'implementation capability to produce correct code edits from detailed specifications. However, p...
Yun Peng, Zi-Han Wu, Ze-Yang Zhuang et al.· 0 citations
Evaluating twelve open and proprietary models, it is found that abstraction failure is lawful, and the central finding is a gap between recognition and cross-voice mapping: models identify a structure within one voice far better than they carry it across voices; every model pays this tax, and mapping strong enough to n...
Tong Zhang, Zhiyuan Shi, Yun Peng et al.· 0 citations
Experimental results on CAIL2018 demonstrate that legal-oriented adaptation based on DRAGIN improves retrieval-augmented reasoning in this Chinese legal judgment prediction setting while maintaining low retrieval overhead.
It is found that defenses rarely improve downstream capability, but instead vary in how they trade safety gains against usability and efficiency as well as performance impact, over-refusal on benign inputs, and inference cost.
TraceProbe is presented, a trajectory-diagnostic framework that recovers what resolve rate hides and adds auditable diagnostic context to outcomes by localizing inspection targets, suggesting failure hypotheses, and prioritizing runs for review.
Rui Shu, C. Chong, Xin Zhou et al.· arXiv.org· 1 citation
SWE-RPG is introduced, a repository-level benchmark that combines executable patch evaluation with validated ground-truth references (GTs) for Requirement Clarification and Implementation Planning, and suggests implicit-requirement recovery as a key candidate direction for improving coding agents.
Xin Zhou, Chun-Yong Chong, Kisub Kim et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.