Formal Verification of Deep Reinforcement Learning Policies with Temporal Logic

Abstract

Deep Reinforcement Learning (DRL) has demonstrated remarkable success in solving complex control problems. However, deploying DRL agents in real-world scenarios, particularly in safety-critical applications, presents significant challenges. The inherent stochasticity and complex decision-making processes of DRL policies often lead to unpredictable and potentially unsafe behavior. This work addresses this challenge by formally verifying the safety and correctness of DRL policies using temporal logic. We construct a temporal logic model that precisely captures the desired behavior of the policy and employ model checking techniques to rigorously determine whether the policy satisfies the specified temporal logic constraints. This approach offers a novel and rigorous method for ensuring the reliability and trustworthiness of DRL systems, bridging the gap between the empirical successes of DRL and the formal guarantees provided by verification techniques. The core contribution lies in applying temporal logic as a precise specification language to define the desired behavior of a DRL policy, enabling automated verification of its adherence to safety and correctness properties. This process moves beyond simply observing policy performance to providing a mathematically sound basis for confidence in the policy's operation.