ExplorIt: Simulation-based Fuzzing for Autonomous Driving Systems via Multi-Agent Interaction Modeling

Abstract

Fuzzing-based simulation testing has become a fundamental technique for assessing the safety of autonomous driving systems (ADS). It operates by iteratively mutating simulation scenario configurations, scheduling scenario execution, and monitoring ADS-involved accidents. However, existing ADS fuzzers commonly rely on simplistic criticality metrics (e.g., inter-vehicle distance) for prioritizing critical scenarios during fuzzing, and lack deterministic guidance about how these heuristic-based critical scenarios should be further mutated to finally induce ADS-responsible accidents. Therefore, these existing fuzzers would inevitably miss truly critical driving scenarios or report various non-ADS-responsible accidents. To address these limitations, our key insight hints that EGO-to-NPC interactions (i.e., those recorded during the execution of a given scenario) offer comprehensive spatial-temporal information for reliable selection of critical scenarios and deterministic scenario mutation. Following this insight, we propose ExplorIt, a simulation-based ADS fuzzer enhanced with systematic formalization of interaction behaviors. Specifically, ExplorIt formalizes the runtime EGO-to-NPC interactions through drivable area estimation. That is, vehicles that share overlapped driving areas are considered to have interactive relationships. Under this modeling, ExplorIt can then deterministically mutate scenarios to curate more critical interactions by making EGO and NPC head for shared drivable areas. Extensive experiments on Apollo 8.0 demonstrate ExplorIt’s effectiveness, revealing 6.7 times more unique ADS-responsible accidents than baseline tools (i.e., DriveFuzz, SAMOTA, and AutoFuzz). Moreover, 83.2% of the reported accidents are ADS-responsible, compared to just 16.8% with baseline tools.