This paper introduces NEUROTESTGEN, a hybrid approach that integrates symbolic execution with LLM-driven test synthesis to generate test cases targeting on-demand code coverage and incorporates an iterative feedback loop that validates LLM-generated tests and provides corrective guidance until the target line or branch...
Rui-Xin Zhang, Jiho Shin, H. Pham et al.· 0 citations
A large-scale empirical study of quality assurance (QA) practices in 157 open-source LLM-based agent projects with at least 100 GitHub stars highlights the need to move beyond feature-level testing toward systematic end-to-end validation that ensures agent workflows remain within intended boundaries when interacting wi...
Wu-Yang Dai, Moses Openja, Jiho Shin et al.· 0 citations
With more than 20,000 Common Vulnerabilities and Exposures (CVEs) reported annually, software vulnerabilities represent a critical cybersecurity challenge. This volume has intensified the demand for automated detection and analysis, motivating the integration of large language models (LLMs) for such tasks. However, exi...
Vishnu Teja Kandalam, Viet Duong, Xiaochang Li et al.· Proceedings of the 32nd ACM...· 0 citations
This paper introduces SecVulEval, a context-aware benchmark designed to evaluate LLMs on vulnerability detection with rich contextual information, and believes it can serve as a foundation for advancing context-aware vulnerability detection with LLMs.
Md Basim Uddin Ahmed, Nima Shiri Harzevili, Jiho Shin et al.· AIware· 2 citations
An evaluation of five state-of-the-art LLMs on OmniVul reveals distinct performance gaps, with top-1 accuracy remaining below 50% on average for vulnerable code detection and CVE identification, demonstrating that current models lack critical reasoning capabilities for reliable vulnerability assessment.
Vishnu Teja Kandalam, Viet Duong, Xiaochang Li et al.· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
