EvoFuzz: IR-Aware Fuzzing for JIT Compilers through the Lens of Node Evolution
Abstract
JIT compilers are core components of modern language virtual machines, and their bugs can crash the VM or silently miscompile programs. Modern JIT compilers perform multiple rounds of optimizations on graph-based intermediate representation (IR), where each round modifies, transforms, or eliminates IR nodes (referred to as node evolution). Bugs arise when the combination of multiple evolution paths violates inherent constraints of the IR. However, existing fuzzing tools lack fine-grained awareness of IR-level node evolution and struggle to construct the compound IR structures required to trigger such evolution, limiting their ability to explore the evolution combination space effectively. We propose EvoFuzz, an IR-aware fuzzing framework for Java virtual machine (JVM) JIT compilers, comprising three components: Node Mutators, which induce IR nodes of different semantic categories through source-level mutations and are portable across compilers; Evolution Profiling, which tracks evolution at the optimization-path granularity and defines a set distance metric as the feedback signal for fuzzing; and Combination-Aware Scheduling, which learns sequential dependencies among mutators via Markov dependencies to optimize mutator selection. We implement EvoFuzz as a practical fuzzing system for production JVMs and evaluate it on multiple versions of OpenJDK HotSpot, OpenJ9, and GraalVM. EvoFuzz discovered 45 bugs, of which 37 have been confirmed and 10 have been fixed by developers. Compared with state-of-the-art JVM fuzzers, EvoFuzz improves code coverage in the HotSpot C2 compiler module by approximately 7%, triggers 1.7–1.9 × as many unique optimization paths on average, and achieves 3.0–3.5 × the baselines’ median set distance at 12 hours. These results show that node-evolution-aware feedback is an effective signal for exposing correctness bugs in production JIT compilers.