Vulnerable Function Detection Using Lexical and Structural Features: An Empirical Study on PrimeVul and DiverseVul

Abstract

A persistent challenge in software vulnerability detection is the failure of many existing approaches to handle highly imbalanced datasets reliably. In practical vulnerability datasets, vulnerable functions usually account for only a small proportion of all samples, which makes model evaluation highly sensitive to feature representation, threshold selection, and class distribution. To address this issue, this study proposes a lightweight feature-based detection pipeline for function-level vulnerability detection. The implemented method combines TF-IDF lexical features with numerical code-statistical features and evaluates several conventional machine-learning classifiers under a unified experimental protocol. PrimeVul is used as the primary dataset, while DiverseVul is introduced for external cross-dataset validation. The experiments include baseline comparison, feature ablation, threshold selection, sensitivity analysis, random-seed stability testing, imbalance-handling evaluation, and cross-dataset assessment. The results show that the proposed feature-based pipeline achieves stable performance under highly imbalanced settings. On the PrimeVul test set, the best configuration achieves an accuracy of approximately 0.9697, an F1-score in the range of 0.26–0.27, and a ROC-AUC above 0.83. The external evaluation on DiverseVul further indicates that the learned feature representation retains a certain degree of cross-dataset generalization. These findings suggest that carefully designed lightweight feature representations, combined with systematic multi-metric evaluation, can provide a reproducible and interpretable baseline for practical software vulnerability detection.