Coverage-Aware Guidance for Novelty-Driven Exploration in Automated Game Testing Under Sparse-Reward 3-D Environments

Abstract

Automated testing of high-fidelity 3D games requires agents to explore large state spaces under sparse rewards and execute context-dependent interactions that expose faults. Random network distillation (RND) promotes exploration by rewarding observations that remain difficult for a predictor network to estimate, but prediction error does not explicitly represent exercised state-action contexts. We propose RND+CAE, which combines RND with coverage-guided adaptive exploration (CAE). CAE maintains visitation counts over environment-adapted coverage abstractions and provides a one-step recovery bonus when structured coverage stops expanding. We evaluate the framework in a randomized maze for spatial exploration and a partitioned arena for interaction-dependent fault discovery. RND remains a strong maze baseline, while RND+CAE achieves comparable cumulative coverage with lower observed variability. In the Arena, RND+CAE obtains the highest mean final unique bug count and bug-discovery area under the curve (AUC) among the main methods. An ablation shows that spatial-only guidance matches Full RND+CAE in final fault breadth but accumulates faults more slowly, whereas removing stagnation recovery reduces bug AUC and the proportion of runs reaching five bugs. These results indicate that geometric coverage strongly influences eventual fault breadth in the fixed Arena and that Full CAE’s clearest additional contribution is sustained late-stage discovery through stagnation recovery. The independent benefits of interaction-oriented keys are not established consistently. The framework may also support reliability validation in complex disaster-response simulations.