A qualitative analysis further identifies six construction patterns in ChatGPT-generated MCQs that contribute to non-functioning distractors and help explain their psychometric characteristics and provide practical guidance for instructors to review and revise ChatGPT-generated items before deployment.
Eng Lieh Ouh, Kar Way Tan, Siaw Ling Lo et al.· ACM Transactions on Computin...· 0 citations
Vulnerability-detection benchmarks score the verdict an agent reaches, not the evidence it gathered. A model that recalls a CVE from pretraining therefore scores the same as one that traced the data flow. We study a task where this difference matters, deciding whether a commit introduces a vulnerability. Instead of sco...
Yi-Kun Li, Jin-Feng Jiang, Yuheng Yieh et al.· 0 citations
VulAgentRL is proposed, an agentic RL framework for interprocedural vulnerability detection built on a Code Property Graph (CPG), which outperforms state-of-the-art baselines, including frontier models, on the strict pair-wise-correct metric while issuing fewer tool calls, and its advantage persists on an out-of-distri...
Yi-Kun Li, Ting Zhang, Jia-Kun Liu et al.· arXiv.org· 2 citations
Experiments on real-world vulnerabilities show that CoSA consistently outperforms function-level and pure-LLM baselines, suggesting that explicit, metric-oriented repository context retrieval is crucial for practical and reliable automated severity assessment.
Jin-Feng Jiang, Yi-Kun Li, Cheng-Ran Yang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
