Skip to content
Review Open access

Deep Reinforcement Learning-Based Intrusion Detection in IoT Networks: A Systematic Mapping and Literature Review

2026 · Computer Modeling in Engineering & Sciences · 0 citations · 126 references

TL;DR

The review reveals that the most used algorithm for DRL-based IDS is Deep Q-Network (DQN), appearing in 8 studies (30.8%), and the most frequently targeted attacks are DoS, DDoS, Backdoors, Mirai, Reconnaissance, Scan, and Torii.

Abstract

: The increasing complexity and heterogeneity of cyberattacks targeting Internet of Things (IoT) environments, driven by the diversity of interconnected nodes and communication channels, necessitate the development of more advanced and intelligent cyber defence techniques. However, the most effective methods are Machine Learning (ML)-based and Deep Learning (DL)-based intrusion detection systems (IDS), which perform well but still face significant limitations and challenges. To address these issues, Deep Reinforcement Learning (DRL) has been proposed in recent years to automatically resolve the issues by detecting attacks in IoT environments. Therefore, this Systematic Literature Review (SLR) presents an up-to-date review by analyzing the existing studies on DRL-based IDS models that detect intrusions in IoT networks. To achieve this goal, this review focuses on and scrutinizes scientific journals and articles extracted from 2020 to 2026 across multiple databases, identifying 267 articles. A systematic mapping procedure was then carried out using Rayyan and Mendeley to screen the articles based on nine well-defined inclusion and exclusion criteria covering publication year, language, publication type, full-text availability, explicit use of DRL techniques, relevance to IoT attack detection, minimum page count, duplication, and open access, which collectively reduced the pool to 26 high-quality studies. The majority of excluded articles 241 in total were removed primarily because they did not explicitly employ DRL techniques in an IoT intrusion detection context, were duplicate records, or did not meet the minimum quality thresholds defined in the eligibility assessment. The review reveals that the most used algorithm for DRL-based IDS is Deep Q-Network (DQN), appearing in 8 studies (30.8%). Three studies utilized feature selection methods, including LightGBM and Mutual Information Feature Selection (MIFS), and the most frequently targeted attacks are DoS, DDoS, Backdoors, Mirai, Reconnaissance, Scan, and Torii. Finally, this research highlights the open issues and challenges for future research in DRL-based IDS models, to enhance IoT network security.

Read PDF

Similar papers

Open access Aug 2026

Proactive Deep Q-Learning Approach for Anomaly Detection in IoT IDSs

An offensive-defensive system based on Deep Reinforcement Learning (DRL) algorithms is proposed, which outperform several state-of-the-art machine learning approaches in the literature, revealing that the systematic incorporation of accurate data engineering and reinforcement learning frameworks generates a field-tested security barrier that offers an expedient reaction to counteract multifaceted threats to IoT networks.

Hawraa A. Habeeb, M. Manaa · 0 citations
Review Open access Jul 2026

A Systematic Review of AI-Driven Intrusion Detection and Performance Optimization in Wireless Sensor Networks

The study analyzes the most recent progress in ML and DL methods used to develop IDS that operate in WSNs through analysis of their primary algorithms and algorithmic combinations and concludes that the DL and hybrid approaches are superior to conventional ML algorithms in handling complicated and imbalanced datasets.

Priyanka Sharma, Mohd. Suhaib Kidwai, Piyush Charan · 0 citations
Review Open access Sep 2026

Deep Learning-Based Intrusion Detection in IoT: A Comprehensive Review of Architectures, Attacks, Challenges, and Future Directions

The rapid proliferation of Internet of Things (IoT) devices across critical domains including healthcare, smart cities, industrial control systems, and intelligent transportation has fundamentally transformed the cybersecurity threat landscape. The inherent characteristics of IoT environments, namely resource-constrained devices, heterogeneous architectures, and large-scale deployment, render traditional Intrusion Detection Systems (IDS) inadequate for the sophisticated and evolving attack vectors targeting these networks. Deep learning (DL) has emerged as a compelling paradigm for next-generation IoT IDS, offering automated feature extraction, temporal pattern recognition, and adaptive threat detection capabilities that address the limitations of conventional approaches. This paper provides a thorough and systematic review of the existing DL methods for IoT intrusion detection. The paper explore the IoT architectural paradigms, outline a four layered taxonomy for types of IoT attacks across its three primary layers Perception, Network and Application as well as Adversarial Machine Learning attacks, and systematically review seven classes of DL architectures Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRU), Autoencoders, Generative Adversarial Networks (GAN), models based on Transformer architecture and Federated Learning frameworks. In a comparative review of forty peer-reviewed studies, we demonstrate that hybrid DL models provide excellent detection performance (99-100% classification accuracy on benchmark datasets) as well as practical viability for deployment with privacy-preserving Federated Learning for large-scale data. The study additionally highlights five enduring challenges class imbalance, adversarial vulnerability, zero-day detection limitations, computational constraints and the absence of standardized benchmarking protocols that together account for the gap between performance benchmarks and real-world deployment efficacy. It outlines future research avenues targeting on five key axes with a particular focus in the integration of Explainable AI (XAI), lightweight edge-deployable architectures, and adversarial robustness mechanisms. This survey identifies a structured reference to advance the state of IoT intrusion detection from research to operationally viable and deployable systems.

Mohammed Gharkan, Mustafa I. Hussien Al-Janabi, Obaid Salim · 0 citations
Review 2026

Deep Learning-Driven Intrusion Detection Systems: A Comprehensive Survey of Architectures, Performance Evaluation, and Research Challenges

A taxonomy of IDS architectures and a survey of recently proposed networks, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), LSTMs, GRUs, Autoencoders, GANs, Transformer-based, as well as hybrids are developed.

Sajith K. V., Gripsy Paul, Bhagavant Deshpande et al. · 0 citations
Open access Aug 2026

Cross-Domain Deep Transfer Learning Framework for Intrusion Detection in Data-Constrained and Resource-Limited IoT Environments

The rapid expansion of the Internet of Things (IoT) has led to the widespread deployment of interconnected smart devices and wireless sensor systems in critical applications, significantly increasing the attack surface for cyber threats. Deep learning (DL)-based intrusion detection systems (IDSs) have demonstrated considerable potential for enhancing IoT security through their ability to automatically learn complex attack patterns and detect anomalous behavior. However, the effectiveness of these approaches often depends on the availability of large volumes of labeled training data, which are difficult to obtain in many IoT environments due to device heterogeneity, evolving attack patterns, privacy constraints, and the limited availability of domain-specific intrusion datasets. Consequently, conventional DL-based IDSs often exhibit reduced performance under data-constrained conditions. To address these challenges, this paper proposes a deep transfer learning (DTL)-based intrusion detection framework for data-constrained and resource-limited IoT environments. The proposed approach leverages knowledge acquired from large-scale computer network intrusion datasets by pre-training deep neural network (DNN) models on source-domain data and subsequently fine-tuning them using smaller IoT intrusion datasets. In addition, pruning and quantization techniques are incorporated to reduce model complexity and enable efficient deployment on resource-constrained IoT edge devices. Experimental results demonstrate that the proposed DTL framework outperforms conventional DL-based IDS models, achieving improvements in accuracy, recall, F1-score, and area under the receiver operating characteristic curve (AUC). Furthermore, the compressed models achieve substantial reductions in model size and improved inference latency. These findings demonstrate the effectiveness of cross-domain knowledge transfer for addressing intrusion detection in data-constrained IoT environments.

Imran Mohammed, Imad Mahgoub · 0 citations
Review Open access Sep 2026

Deployment-oriented evaluation of deep learning network intrusion detection systems

Deep learning models are now used as a security layer in network intrusion detection systems. These models are often used in Internet of Things (IoT), industrial Internet of Things (IIoT), software-defined networking (SDN), and future internet communication environments. Most of the models reviewed in literature are still trained and tested on the same dataset under closed-set evaluation conditions; these models are also required to tolerate traffic distribution changes, unseen attacks, and uncertain predictions. Review articles from 2024 to 2026 focus on three main concerns, namely, cross-dataset generalization under domain shift, open-set intrusion recognition, and reliability through calibration or uncertainty-aware evaluation. Many of the reviewed models have been developed using isolated datasets and assumptions that weaken direct comparisons among them. This makes it difficult to judge deployment readiness across other environments. Most studies in literature address only one of these concerns instead of all three; across 29 empirical studies published between 2024 and 2026, none have evaluated all three concerns together. Several of these studies report high benchmark accuracies, but the reviewed corpus lacks joint evidence that the models can handle new attacks and new datasets while remaining reliable in IoT, IIoT, SDN, and future internet environments. Future works therefore require better cross-dataset testing, more evaluations of unseen attacks, and reliability checks in real-world secure communication environments.

Unknown authors · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.