Overall, DGFuzz turns discovered defect distributions into reusable feedback for automated DNN testing and improves unique-defect discovery under limited testing budgets.
Abstract
Deep neural networks (DNNs) are increasingly used in safety-critical software systems, where incorrect behavior can cause severe consequences. Fuzz testing is an effective technique for automatically exposing such behaviors by generating mutated inputs, but existing DNN fuzzers mainly use neuron-level, critical-neuron, or gradient-level feedback. They rarely exploit the spatial distribution of defects already found during testing. This limits testing efficiency because adversarial defects often concentrate in vulnerable regions of the feature space rather than appearing randomly. This paper presents DGFuzz, a defect-guided directed fuzz testing framework for DNNs. DGFuzz maps discovered adversarial defects to the penultimate-layer feature space and identifies defect-concentrated clusters as vulnerable regions. These vulnerable regions are then reused as testing feedback through two strategies: defect-cluster-guided seed initialization, which selects seeds closer to high-risk regions, and adaptive population scheduling, which allocates mutation resources according to regional defect-discovery potential. We evaluate DGFuzz on 9 DNN models over MNIST, CIFAR-10, and GTSRB and compare it with representative neuron-coverage-, critical-neuron-, and gradient-guided fuzzing methods. DGFuzz discovers more unique defects in 8 of 9 models under a fixed mutation budget and achieves higher fuzzing-phase efficiency than the baselines. Extended-budget and ablation studies further quantify the contributions of vulnerable-region modeling, seed initialization, and adaptive scheduling. Overall, DGFuzz turns discovered defect distributions into reusable feedback for automated DNN testing and improves unique-defect discovery under limited testing budgets.
Greybox fuzzing is a widely used technique for automatically discovering vulnerabilities in programs. Existing greybox fuzzing approaches primarily rely on runtime coverage feedback to guide testing, continuously exploring program paths to uncover defects. However, using a single coverage metric sometimes fails to effectively guide testing toward deeper program execution. Moreover, fine-grained coverage metrics can lead to excessive seed corpus expansion, reducing efficiency. To address these limitations, we propose Cohort, a novel greybox fuzzing technique that combines multiple coverage metrics and selectively applies them to high-priority regions with unsolved constraints. During runtime, Cohort maintains a precise list of basic blocks associated with unsolved constraints, and concentrates various coverage feedback mechanisms on these blocks, since solving their constraints is crucial for deeper exploration. To sustain efficiency, Cohort continuously prunes seed corpus by retaining only those seeds that contribute useful coverage information. In addition, Cohort dynamically recompiles the target program to remove redundant instrumentation and identifies critical byte regions in inputs, thereby improving performance while maintaining low overhead. Experimental results show that Cohort outperforms state-of-the-art fuzzers, achieving an average of 24.6% higher edge coverage and 30.6% higher vulnerability detection rate. Furthermore, Cohort discovered 8 previously unknown vulnerabilities, with 8 CVEs assigned.
Xu Chen, Zhengkun Pan, Ruoming Fang et al.· ACM Transactions on Software...· 0 citations
NeuronFuzz is presented, a white-box fuzzing framework that exploits internal safety neurons as continuous execution feedback for LLM safety evaluation and achieves a 76-100% jailbreak discovery rate, outperforming baselines by up to 48 percentage points.
Zhiyuan Xu, Muhammad Firhard Roslan, Joseph Gardiner et al.· 0 citations
With the widespread deployment of deep neural networks (DNNs) in safety-critical domains, reducing the cost of model validation under limited testing budgets has become increasingly important. Existing test case prioritization techniques often rely on single-checkpoint confidence signals derived from output probabilities. However, DNNs can be confidently wrong, and the confidence margin between the predicted and competing classes is frequently small, which weakens early fault discovery. To address this limitation, we propose a Neural-Collapse-Inspired Prioritization (NCIP) framework that replaces absolute confidence with cross-checkpoint prediction variability in the terminal training regime, where model geometry becomes highly structured. NCIP introduces two key components. First, it selects an NC-guided representative subset of training checkpoints using an equiangularity score of classifier weights, quantified as the standard deviation of pairwise cosine similarities among class weight vectors. Second, it prioritizes test inputs by their prediction variability across the selected checkpoints, surfacing boundary-adjacent and failure-prone samples that are unstable under checkpoint-induced decision boundary shifts. Extensive experiments across multiple datasets and architectures show that NCIP achieves strong performance in early fault discovery compared with competitive baselines, with 1.5 to 16.6 percent RAUC-ALL gains and 4.9 to 20.6 percent RAUC-500 gains under the same testing budget. NCIP further attains the best average performance across all dataset-model pairs.
Chunyu Liu, Mingyuan Li, Yang Li et al.· arXiv.org· 0 citations
VeRe is proposed, a verification-guided repair framework that leverages linear relaxation to precisely and efficiently estimate the repair significance of neurons and synthesizes ideal intervals that provide sound guarantees for correct behaviors, thereby facilitating surgical and targeted adjustments of neuron parameters to rectify faulty behaviors.
Jia-Nan Ma, Wei Chen, Pengfei Yang et al.· ACM Transactions on Software...· 0 citations
Deep learning libraries underpin many safety- and reliability-critical applications, yet existing API-level testing techniques often rely on intra-library properties or CPU--GPU differential oracles and may miss defects that behave consistently across hardware backends. We present Xamt, a cross-framework differential fuzzing approach for deep learning library APIs. Xamt constructs and tests execution-validated groups of APIs intended to implement equivalent operations across seven libraries. It uses explicit API aliases and parameter-role normalization to construct candidate correspondences and validates them through pairwise execution and a group-level behavioral check on canonical ordinary inputs. The resulting groups are explored using variance-guided differential fuzzing with ordinary, boundary, and non-finite inputs. Crash and inconsistency oracles flag executions exhibiting abnormal termination or inconsistent outputs for subsequent reproduction and analysis. Across the seven libraries, Xamt constructs 676 execution-validated groups containing 2,563 matched APIs. Among these, Xamt identifies 72 independently reproduced discrepancy cases, including 4 crash cases and 68 output inconsistencies. Among the 72 developer reports, 25 have been confirmed, including 23 that have been fixed.
Bin Duan, Rui Dong, Naipeng Dong et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
