VistaFuzz is introduced, a document-guided fuzzing technique that uses a locally served open-sourced LLM to extract parameter specifications from API documents and generate inputs that satisfy both parameter constraints and inter-parameter dependencies.
Abstract
Python libraries underpin deep learning, scientific computing, data analysis, and computer vision, making their reliability critical to downstream applications. Testing their APIs requires inputs that satisfy both per-parameter constraints and dependencies among parameters. Existing approaches either leave such constraints implicit in generated programs or rely on library-specific parsing rules. This paper introduces VistaFuzz, a document-guided fuzzing technique that uses a locally served open-sourced LLM to extract parameter specifications from API documents and generate inputs that satisfy both parameter constraints and inter-parameter dependencies. We evaluate VistaFuzz on 7,718 APIs across twelve Python libraries. Inter-parameter relationships occur in 40.1\% of tested APIs, and disabling their resolution reduces the valid generation rate on those APIs from above 95\% to 31.6\%--52.8\%. VistaFuzz reports 74 issues, of which 43 have been confirmed by developers and 29 have been fixed.
Fuzzing Rust library APIs requires constructing well-typed, compilable call sequences that satisfy ownership rules, generic parameters, and trait bounds; existing tools ignore these constraints or use shallow heuristics, yielding low coverage. We present GRAFT, which extracts structured API information from Rust documentation, builds an API dependency graph via recursive generics-aware type matching, and uses topology-guided traversal plus LLM synthesis with compiler-error feedback to produce compilable fuzz targets. On 13 crates from crates.io, GRAFT achieves 80.75% macro-average API coverage at 96.19% compilation success, outperforming RULF and RPG by 4.76x and 2.43x, and reaching 1.41x the average API coverage of deepSURF on crates with unsafe-reaching APIs.
Yiming Chen, Kaiwen Zhang, Guanjun Liu et al.· 0 citations
Due to their black-box nature, LLMs suffer from limited explain- ability and a lack of determinism. Their usage cost can also rise, particularly with repetitive tasks on large codebases. To mitigate this, we conduct a novel empirical study targeting three domain- specific languages for transformation rules, namely Comby, GritQL, and Ast-Grep. We evaluate three LLMs (GPT-5.4, GPT-oss-120B, and Llama3.1-8B) on six diverse datasets covering four software- evolution tasks: API misuse correction, program repair, API migra- tion, and language version migration. Our results provide evidence that transformation rule synthesis moves beyond proof-of-concept with strong frontier models. GPT-5.4 achieves consistently high rule applicability rates and produces transformations closest to the ground truth across most benchmarks. Smaller and open-weight GPT-oss-120B and Llama3.1-8B models remain effective for simpler, localized changes but struggle with complex migration scenarios. We also observe non-negligible generalizability through the usage of meta-variables and through a high reuse score in the first quartile of many datasets. Finally, when compared to the anti-unification algorithm, LLMs outperform it in correctness, but underperform in rule applicability. Overall, our results show great potential for LLMs to generate sound, correct, generalizable, and reusable rules.
Axel Allain, Aymeric Blot, D. Khelladi et al.· 1 citation
These findings provide the largest empirical characterization of PBT in Python to date, highlight developers’ difficulties in adopting the technique, and expose limitations of current tool support.
Isadora de Oliveira, Arthur Lisboa Corgozinho, Henrique Rocha et al.· Empirical Software Engineeri...· 0 citations
Differential compiler testing requires automatically generated programs that are not only diverse and bug-revealing, but also semantically well-defined and reproducible. Rule-based generators provide strong validity guarantees but offer limited control over semantic variation, while large language models (LLMs) can synthesize expressive programs without principled mechanisms for balancing competing testing objectives. This paper proposes LMOEC, a constrained multi-objective evolutionary framework that integrates code language models as semantic genetic operators within an NSGA-II search process. Instead of using the LLM as a one-shot generator, we employ it for population initialization, crossover, and mutation at the program level, enabling semantics-aware recombination while preserving strict admissibility constraints. Compiler test generation is formulated as a multi-objective optimization problem that simultaneously promotes structural diversity, cross-configuration output inconsistency, semantic complexity, and robustness to mutation. A constraint-driven acceptance pipeline enforces syntactic validity, deterministic execution, bounded runtime, and avoidance of undefined behavior before evolutionary selection. By maintaining a Pareto front of non-dominated programs, LMOEC preserves multiple high-value test archetypes reflecting different trade-offs between bug exposure and reproducibility. The framework demonstrates how expressive code models can be systematically embedded into evolutionary multi-objective optimization for reliability-critical software testing.
Lang Hong Nguyet Anh, Ho Viet Duc Luong, Vu Van An· Annual Conference on Genetic...· 0 citations
The paper investigates the possibility of predicting function-inlining decisions in compact compilers, source-to-source tools, and interpreters where the reuse of GCC or LLVM optimization infrastructure is impractical and proposes a portable inlining-prediction framework that can be emitted as ordinary C code without a compiler-runtime dependency.
F. Nikolai, Vinarsky Alexander· arXiv.org· 0 citations
Existing fuzzers for PDF readers rely on simple test cases that involve only individual API calls, leading to limited coverage and potentially missing vulnerabilities that require sequences of API calls. To address these limitations, we propose PDFuzzer, a novel PDF engine fuzzer that automatically generates complex and meaningful API call sequences. PDFuzzer first uses a Large Language Model (LLM) to construct context-free grammars and infer the relationships between individual API calls from specifications extracted from JavaScript API manuals and execution traces. Based on the grammars and relationships, PDFuzzer employs a constraint solver to generate concrete API call sequences for fuzzing. Our experiments show that PDFuzzer significantly outperforms state-of-the-art PDF fuzzers (TypeOracle, Favocado, and Cooper) and LLM-based fuzzers (Fuzz4All, naive LLM) on three mainstream PDF readers: Adobe Acrobat Reader, Foxit PDF Reader, and PDF-XChange Editor. PDFuzzer achieves up to 48% higher coverage than existing tools and identifies 31 zero-day vulnerabilities in these readers, from information leakage to arbitrary code execution. Our ablation study validates the necessity of each component, including LLMs, which achieve high accuracy across all pipeline stages (93-98%). We disclosed all vulnerabilities to the vendors via a coordinated vulnerability disclosure process and received bug bounties.
Suyue Guo, Stijn Pletinckx, Tianle Yu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
