Formal Verification of Deep Reinforcement Learning Policies via Abstract Interpretation

Abstract

Deep Reinforcement Learning (DRL) has achieved remarkable success in various domains, including game playing and robotics. However, the inherent uncertainty and complexity of DRL policies pose significant challenges to safety and reliability. This work introduces a novel approach to formally verify the safety and robustness of DRL policies using abstract interpretation. We leverage abstract domains to represent the learned value function, enabling the detection of potential safety violations without requiring actual execution of the policy. The core idea is to transform the continuous value function, typically output by a DRL agent, into a discrete abstract representation. This allows us to apply well-established abstract interpretation techniques to identify potential issues like out-of-bounds access, negative rewards, or violations of specified constraints. The technique offers a practical method for assuring the correctness of DRL policies, particularly in safety-critical applications.