VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection

VICBench: A Multi-Language Benchmark for Code Vulnerability Detection

VICBench enables robust evaluation of vulnerability detection approaches and shows that state-of-the-art algorithms V-SZZ and LLM4SZZ achieve only 33.3%-40.1% F1, confirming that using existing approaches still entails significant manual effort.

From Trace to Line: An Empirical Study of What Drives LLM-Based OSS Vulnerability Localization

This paper introduces T2L (Trace-to-Line), a reproducible research framework that narrows repository-scale code into candidate vulnerable lines through AST-based chunking, structured diagnostic information collection, and evidence-guided refinement that improves trace-to-line localization.

VulRESC: A vulnerability detection framework based on risk path extraction and inter-procedural semantic completion

Software vulnerability detection increasingly relies on learning-based models. However, most existing methods analyze individual functions in isolation, making it difficult to capture vulnerabilities caused by cross-function calls; directly introducing complete call chains can also lead to context expansion and noise accumulation. This paper proposes VulRESC, a vulnerability detection framework based on risk path extraction and interprocedural semantic completion. The method first constructs code property graphs and extracts call paths related to external inputs and high-risk operations through a riskpoint- driven pruning strategy. It then uses large language models to generate structured summaries for callees along the sequence in a bottom-up manner and introduces a variable-name alignment mechanism to bridge the semantic gap across function boundaries. Finally, the proposed DualVulBERT dual-stream model is employed to jointly identify vulnerabilities by fusing source code features and inter-procedural semantic summaries through cross-modal attention. Experimental results on real-world C/C++ vulnerability datasets show that VulRESC achieves an accuracy of 68.03% and an F1-score of 69.37%, outperforming representative existing methods.

Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale

Language-model agents increasingly operate over complete software repositories, yet cybersecurity evaluations primarily measure whether they can detect, reproduce, or repair vulnerabilities rather than whether they can locate the relevant code. We study vulnerability localization: given a weakness class and an unfamiliar repository, identify the implementation files associated with that weakness. We introduce the Vulnerability Localization Benchmark (VLoc Bench), comprising 500 real world vulnerabilities from 290 repositories across six package ecosystems and 147 CWE categories. Each task pairs repository snapshots immediately before and after a security fix. On the vulnerable snapshot, an agent receives only the CWE description and read-only terminal access and must return the affected files; on the patched snapshot, it must determine that the recorded vulnerability is no longer present. We evaluate 27 language models and four static-analysis tools under a common agent interface. Repository-scale vulnerability localization remains difficult: the strongest system achieves 0.229 File F1, and 38.4% of tasks receive no correct localization from any evaluated model. We further find that stronger localization does not imply reliable behavior after remediation: systems that identify vulnerable files effectively can still report unsupported locations on patched repositories. These results establish vulnerability localization as a distinct repository-scale capability and provide a setting for studying both how security agents search for vulnerable code and when they should refrain from reporting it.

HSF-Vul: hierarchical semantic fusion for vulnerability detection

Probing the Prefill: Detecting Code Vulnerabilities via Latent Activations

LLM-based code generation is now embedded in mission-critical pipelines, but defenses against vulnerable output remain post-hoc -- static analyzers, fine-tuned classifiers, or an LLM judge that screen completed code, ignoring the generating model's own internal state. We test a narrower, directly measurable question: when an LLM reads a piece of C/C++ code as context, do its hidden activations already carry a signal about that code's vulnerability status? We extract last prefill token activations from four LLMs (Granite-4.1-8B, Qwen3.5-9B, Qwen3.6-27B, Gemma-4-12B) across three model families and train MLP probes on these activations. We evaluate them on four function-level C/C++ benchmarks (Devign, Big-Vul, Draper VDISC, PrimeVul). Our probes achieve 41.7\% average F1 using 13.4--16.0M-parameter probes -- under 0.2\% of base-model size. On Devign, the best probe (Qwen3.5-9B, 68.8\% F1) matches the published fine-tuned-classifier SOTA (67.9\%) despite reading only a frozen, general-purpose LLM's activations; on the harder, more imbalanced benchmarks (Big-Vul, Draper VDISC, PrimeVul) probes trail SOTA substantially. This is early evidence that a coding LLM's own representation of arbitrary code is informative about that code's vulnerability status, motivating further work toward lightweight, model-native vulnerability screening.