Multi-Agent Reinforcement Learning System for Autonomous Network Intrusion Response

Abstract

Cyber threats are becoming so complicated and rampant in today's highly connected digital world that there is a tremendous need for intelligent, adaptive IDS. Detecting and responding against network intrusions in real-time is the aim of the present work, which proposes a real-time Network Intrusion Detection and Automated Response Framework developed by Multi-Agent Deep Q-Network (MADQN). The proposed design uses the UNSW-NB15 network traffic set and discriminates and classifies attacks by using supervised machine learning models including Logistic Regression model, Decision Tree model, Random Forest model and the Neural Network classifiers. To augment the automation of the response, a reinforcement learning layer of three cooperative agents: Detection Agent, Blocking Agent and Isolation Agent is introduced based on Deep Q-Networks (DQNs). They learn the optimal mitigations actions they should take on a network state represented in 221 dimensions, and through the experience replay and epsilon-greedy exploration with the reinforcement learning, the agents learn. The results of the experiments demonstrate excellent response detection and 99.38% classification accuracy for the Random Forest model; for the intelligent response selection, the MADQN framework has a good neutralization rate of 93.94%. The architecture designed in the Flask can be fully integrated to the user authentication, attack prediction, attack response management and historical monitoring architecture. Overall, the proposed framework presents a powerful cybersecurity solution that integrates ML and MARL, enabling more secure and resilient networks.