Skip to content
Open access

Adversarial Vulnerabilities in Cooperative Multi-Agent Reinforcement Learning for Distributed 5G Security

Aug 2026 · Scientific Journal of Informatics · 0 citations · 34 references

TL;DR

This study provides among the first empirical evaluations of adversarial fragility in cooperative MARL-based intrusion detection within distributed 5G-oriented security abstractions, demonstrating that cooperative intelligence alone does not guarantee adversarial robustness.

Abstract

Purpose: This paper focuses on examining the robustness of a cooperative Multi-Agent Reinforcement Learning (MARL)-based Intrusion Detection System (IDS) for intrusion detection in decentralised 5G security settings. Even though MARL techniques have proven effective against dynamic threats in decentralized 5G networks, current research has not considered any adversarial scenarios at all. Methods: A cooperative MARL-based Intrusion Detection System was developed through the CRISP-DM approach. Radio Access Network (RAN), MEC, and Core agents were trained using Centralised Training with Decentralised Execution (CTDE) and Deep Q-Network (DQN) methods. The algorithm was tested on the NSL-KDD and UNSW-NB15 datasets against Fast Gradient Sign Method (FGSM) evasion attacks (ε = 0.05-0.30) and Byzantine poisoning attacks with 5%, 10%, and 20% compromised agents. Result: The model achieved 96.94% accuracy on NSL-KDD and 85.15% on UNSW-NB15 in clean scenarios. The FGSM attack at ε = 0.20 resulted in substantial performance deterioration, leading to accuracy drops of 50.14 and 45.26 percentage points, respectively, and a simultaneous increase in false positives. Byzantine poisoning produced smaller but persistent decreases in accuracy of 12.03 and 2.62 percentage points, respectively. Novelty: This study provides among the first empirical evaluations of adversarial fragility in cooperative MARL-based intrusion detection within distributed 5G-oriented security abstractions, demonstrating that cooperative intelligence alone does not guarantee adversarial robustness.

Read PDF

Similar papers

Open access Jul 2026

Detecting adversarial evasion in deep learning intrusion detection systems using explainable AI

Deep learning based network intrusion detection systems (IDS) can achieve strong traffic classification performance, but their resilience to adversarial manipulation remains a critical concern. This study evaluates the adversarial robustness of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models in a multiclass intrusion detection setting using the Train_Test_Network dataset with ten traffic classes. The models were trained on true sliding flow-window sequences under a unified preprocessing pipeline to support fair comparison. Adversarial robustness was first assessed under a white-box Fast Gradient Sign Method (FGSM) setting and then broadened through additional FGSM and Projected Gradient Descent (PGD) stress testing. SHapley Additive exPlanations (SHAP) were further used to analyse explanation instability under clean and adversarial conditions, and explanation-drift features were evaluated as a secondary adversarial detection signal. Under clean evaluation, both models achieved strong and nearly identical performance, with accuracies of 0.9614 for LSTM and 0.9615 for GRU and weighted F1-scores of 0.9597 and 0.9598, respectively. Under the main FGSM condition, performance declined substantially: the LSTM achieved adversarial accuracy of 0.6094 and weighted F1-score of 0.6290 with an evasion rate of 37.38%, while the GRU achieved adversarial accuracy of 0.5130 and weighted F1-score of 0.5690 with an evasion rate of 47.02%. The broader robustness sweep showed that iterative PGD exposed stronger fragility than FGSM alone. SHAP analysis indicated that adversarial perturbation altered both prediction outcomes and local explanation structure. A learned explanation-driven detector improved over the rule-based baseline, while larger-scale validation confirmed that explanation drift remained informative, though not perfectly separable, at broader scale. Overall, the results show that strong clean performance does not imply adversarial robustness, and that explanation drift provides a useful auxiliary signal for adversarial monitoring in recurrent IDS models.

Elijah M. Maseno, Yanxia Sun, Zenghui Wang · 0 citations
Conference Jul 2026

An Adaptive Defense Framework for Enhancing Adversarial Robustness in Deep Learning-based Network Intrusion Detection Systems

This study investigated the robustness of deep learning-based Network Intrusion Detection Systems (NIDS) against adversarial attacks by proposing a confidence-aware adaptive defense framework. The proposed approach integrates a baseline feedforward neural network, an adversarially trained robust model, and an adversarial detector to dynamically select the most appropriate prediction path based on detector confidence. Experimental evaluation under single-step, multi-step, and adaptive adversarial attack scenarios demonstrated that the framework significantly improves detection robustness while maintaining high classification accuracy on clean network traffic. The adaptive fusion strategy effectively mitigates the impact of adversarial perturbations, reducing misclassification rates and enhancing the reliability of intrusion detection in dynamic cybersecurity environments. These findings confirm that confidence-guided adaptive defense mechanisms provide a practical solution for strengthening the resilience of AI-driven NIDS against evolving attack strategies. However, the proposed framework was evaluated using controlled experimental settings and specific attack models, which may not fully represent the diversity of real-world cyber threats. Future work will focus on validating the framework in large-scale operational networks, extending it to advanced zero-day and adaptive attacks, and investigating lightweight deployment strategies for real-time edge and cloud-based cybersecurity applications.

Aastha Ahlawat, Anurag Goel · 0 citations
Conference Jul 2026

Multi-Agent Reinforcement Learning System for Autonomous Network Intrusion Response

Cyber threats are becoming so complicated and rampant in today's highly connected digital world that there is a tremendous need for intelligent, adaptive IDS. Detecting and responding against network intrusions in real-time is the aim of the present work, which proposes a real-time Network Intrusion Detection and Automated Response Framework developed by Multi-Agent Deep Q-Network (MADQN). The proposed design uses the UNSW-NB15 network traffic set and discriminates and classifies attacks by using supervised machine learning models including Logistic Regression model, Decision Tree model, Random Forest model and the Neural Network classifiers. To augment the automation of the response, a reinforcement learning layer of three cooperative agents: Detection Agent, Blocking Agent and Isolation Agent is introduced based on Deep Q-Networks (DQNs). They learn the optimal mitigations actions they should take on a network state represented in 221 dimensions, and through the experience replay and epsilon-greedy exploration with the reinforcement learning, the agents learn. The results of the experiments demonstrate excellent response detection and 99.38% classification accuracy for the Random Forest model; for the intelligent response selection, the MADQN framework has a good neutralization rate of 93.94%. The architecture designed in the Flask can be fully integrated to the user authentication, attack prediction, attack response management and historical monitoring architecture. Overall, the proposed framework presents a powerful cybersecurity solution that integrates ML and MARL, enabling more secure and resilient networks.

S. Banu, D. Manogna, Farooq Sunar Mahammad et al. · 0 citations
Open access Jul 2026

Reinforcement Learning Approach for Cybersecurity Threat Detection

Traditional intrusion detection systems (IDS) struggle to detect evolving cyber threats due to their reliance on static signatures and fixed decision boundaries. Existing machine learning-based approaches partially address this limitation but often fail to generalize to zero-day attacks and lack adaptability in dynamic network environments. To address these challenges, this paper proposes a reinforcement learning-based intrusion detection system (RL-IDS) that model’s detection as a sequential decision-making problem using flow-level telemetry. The framework is implemented on the CIC-IDS2017 dataset with an isolated zero-day partition within a custom OpenAI Gym environment, incorporating asymmetric reward design, curriculum learning, entropy annealing, and early stopping to train Q-learning, Deep Q-Network (DQN), and Proximal Policy Optimization (PPO) agents. Experimental results show that the PPO-based RL-IDS achieves an F1-score of 0.857 with less than 4% false positives on known attacks, outperforming both DQN and a 400-tree Random Forest baseline. More importantly, it detects 27.7% of previously unseen zero-day attacks (Heartbleed and Infiltration), where the Random Forest fails completely. The system also processes over 290,000 flows per second, demonstrating real-time feasibility. These results demonstrate that reinforcement learning enables a practical balance between accuracy, adaptability, and efficiency, making it a promising solution for next-generation intrusion detection systems.

M. Sayeed, Ennbaraaj G. Sundharajan, Golam Md Mohiuddin · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.