ACID: beta-testing active inference for active cyber-defence

Abstract

Active cyber-defence employs anticipatory techniques to proactively mitigate cyber threats. Among such techniques, the most relevant one consists in the continuous updating of cyber-threat intelligence. In this respect, there is a trade-off between the cost of intelligence assets and the benefits derived from their exploitation. While the optimisation of this trade-off has recently been tested with conventional reinforcement learning, this paper investigates the suitability of active inference-based agents for addressing the same challenge. A novel mathematical formulation for Active Inference over fully-observable Markov Decision Processes is provided in this paper, and four different off-policy bootstrapped neural implementations are given. Empirical evaluation highlights that deep active inference agents not only achieve a competitive performance with conventional deep reinforcement learning systems, but also that, crucially, some formulations of epistemic gain can indeed foster exploration and provide a discernible performance boost in terms of return maximisation. This work contributes to understanding the empirical boundaries of Active Inference’s unique mechanisms in complex, real-world-inspired environments, informing future architectural design for deep active inference agents.