Towards maintainable AI-driven network anomaly and threat detection: a comparative analysis of datasets, preprocessing techniques, and model trade-offs
Aug 2026· Artificial Intelligence Review· 0 citations
TL;DR
A comparative experimental study of anomaly and threat detection techniques used in network analysis through a multistep pipeline, demonstrating that hybrid architectures achieve superior generalisation, yet face challenges regarding computational overhead and cross-dataset adaptability.
Abstract
As the integration of Artificial Intelligence into network intrusion detection systems matures, a critical gap remains in the rigorous empirical benchmarking of datasets, preprocessing techniques, and model effectiveness. This article presents a comparative experimental study of anomaly and threat detection techniques used in network analysis through a multistep pipeline. First, we perform a structured comparison and Exploratory Data Analysis of the most commonly used network security datasets to quantitatively assess their balance, diversity, and real-world representativeness. Subsequently, we experimentally evaluate the performance of distinct Machine Learning, Deep Learning, and Hybrid Models derived under standardized preprocessing techniques to determine the most effective combinations for specific attack vectors. Our results demonstrate that hybrid architectures achieve superior generalisation, yet face challenges regarding computational overhead and cross-dataset adaptability. Addressing these limitations, we propose a proof-of-concept adaptive architecture designed to handle concept drift and adversarial threats. Finally, we outline a roadmap for future research, emphasizing the necessity of dynamic, verifiable AI-driven systems that operate reliably in evolving cybersecurity environments.
This study evaluates the generalization capability of models such as LGBM, RF, XGB, and LSTM, particularly in identifying previously unseen attacks, and investigated the impact of feature selection on generalization and examined how performance changes when combining different datasets.
Miguel Silva, J. Vitorino, Daniela Pinto et al.· International Conference on...· 0 citations
The wide-scale uptake of machine learning applications in safety-sensitive applications renders modern AI deployments prone to adversarial attacks, statistical distribution changes, and various governance reliability issues. Current AI security methodologies generally handle the discussed issues separately, making the current security approaches ineffective in real-world conditions. In this work, an integrated AI security pipeline combining the functionalities of statistical drift detection, adversarial robustness evaluation, governance auditing, and experiment tracking is suggested. The presented system uses Kolmogorov—Smirnov tests, Population Stability Index analysis, and drift detection in data streams via ADWIN in addition to adversarial robustness evaluation through FGSM, PGD, and DeepFool attacks. The Giskard library was used to audit AI models’ performance from the governance perspective. Evaluation of our approach on image and tabular datasets showed the capability of detecting statistically significant drift and significant CNN robustness degradation under increasingly complex adversarial attacks. It turned out that iterative and geometry-aware attack schemes perform significantly better than one-shot perturbations. Drift detection proved effective at spotting statistically significant distribution changes before actual deployment failures.
Siddharth Kumar, Siddhanth Harish Bist· AI Engineering· 0 citations
Cloud computing has emerged as an important core to the contemporary digital services, facilitating scalable, on demand provisioning of resources across a variety of application fields. Nevertheless, this multi-tenant and dynamic environment of clouds and the amplified attack surface make the detection of intrusions through reliable methods a consistent issue that cloud security systems struggle with. The proposed work is a Generative Adversarial Network (GAN)-based hardening framework of cloud intrusion detection systems, targeting better resilience to changing and low-rate cyberattacks. The methodology combines a conditional generator which is used to generate realistic cloud-specific attack traffic, a discriminator used to refine the adversarial traffic, as well as a co-trained intrusion classifier trained on both clean and synthetic data in a closed-loop way. The feature-aware regularization is introduced to maintain the statistical consistency of network traffic, and optimize the attack diversity. The proposed approach is proved to yield better results in comparison with signature-based, machine learning, deep learning, and adversarial ML-based IDS models by experimental assessment. Significant gains in the accuracy of identifying, the ability to recall, stability, and minimizing errors are also noticed with quantifiable increases observed in all evaluation measures. These findings represent the usefulness of adversarial data-driven learning to develop robust, adaptive, and future-ready cloud intrusion detection systems.
T. Divya, Sheik Saidhbi, S. Umarani et al.· 2026 International Conferenc...· 0 citations
Deep learning provides better precision to intrusion detection systems, but the so-called black-box character of these models compromises trust. This paper offers a comparative framework of XAI methods assessment, and bridges standardized metrics and robustness testing loopholes. We include an evaluation methodology that uses fidelity, stability, latency, and robustness measures; an experimental study that compares SHAP and LIME on a BiLSTM model that is trained on CIC-IDS2017 on six attack types; and the first systematic measure of robustness of XAI explanations to adversarial perturbations. Findings indicate that both approaches obtain fidelity of over 0.92 with SHAP being 23 and 18 percent more stable and robust respectively in adversarial settings, though with 5.7 times higher latency. The quality of explanations depends on the attack. These results give practical recommendations on the selection of XAI and point out weaknesses in existing methods of explanation.
Abdulrahman Nassar, Mohammad Alkhazaleh, Musab B. Alzghoul· IEEE Jordan Conference on Ap...· 0 citations
An intelligent DDoS detection and mitigation framework that combines classical Machine Learning (ML) classifiers with Deep Learning (DL) architectures to achieve high-fidelity, low-latency attack identification across heterogeneous network topologies is presented.
S. Singh, Alok Kumar· International Journal of Com...· 0 citations
An explainable deep learning framework evaluated across multiple heterogeneous cyber attack datasets, including Kitsune, Server-Based network data,enterprise logs, and Malware Traffic datasets, demonstrating the effectiveness of the proposed framework in handling heterogeneous network traffic while providing interpretable insights into model predictions.
Abed Alanazi· Scientific Reports· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
