Bridging the Simulation-to-Emulation Gap in Autonomous Cyber Defence: A Hybrid PPO and LLM-Based Framework

Abstract

Autonomous cyber defense has become an important research area as modern networks increase in complexity, size, and vulnerability to rapid multi-stage attacks. Although reinforcement learning shows potential for making defensive decisions, many current studies focus only on simulations. This leaves questions about how well trained policies work in more realistic settings. Practical autonomous defence systems also require not only adaptive action selection but enforcement of safety constraints and operator-facing explainability. This paper presents a hybrid framework bridging simulation and emulation. We train a Proximal Policy Optimisation (PPO) defender in the CybORG CAGE Challenge 2 and evaluate it within a Dockerbased network emulation using a custom telemetry bridge. The system integrates the PPO policy with a rule-based guard layer to prevent unsafe actions and a Large Language Model (LLM) to provide real-time natural language explainability to operators. Experimental results show that the trained policy successfully transfers to the Docker environment, though its strategic behaviour adapts noticeably to live emulation conditions. Overall, this work demonstrates that combining reinforcement learning with rule-based logic and LLM explainability enables autonomous defenders to successfully transition into live environments.