Robustness Evaluation and Detection of Transferable Adversarial Attacks in ML-Based NIDS

Abstract

Machine learning-based network intrusion detection systems (ML-based NIDS) are vulnerable to adversarial evasion, where malicious samples are perturbed to evade detection and be misclassified as benign. Despite growing research on adversarial attacks and defenses for ML-based NIDS, comparative evaluations of multiple attack types, detection models, and defense strategies under a common setting remain limited. In this paper, we evaluate eight adversarial evasion attacks, fifteen detection models, and three representative defense strategies using the NF-UQ-NIDS dataset, which includes recent traditional and IoT network traffic with twenty distinct attack categories. The evaluation compares model performance on clean test data and on robustness evaluation sets that include adversarial samples, analyzes attack success consistency across models, and examines the effect of defense strategies on adversarial robustness. To support model comparison, we introduce the Robustness Index (RI), a compact comparative metric that rewards high balanced accuracy and macro-F1 score computed on the robustness evaluation set while penalizing high attack success rate (ASR). We further present AR-NIDS, a two-stage framework that uses an adversarially trained ensemble to distinguish normal, attack, and adversarial samples, followed by an adversarial attack classifier to identify the attack type. Under the evaluated transfer-based setting, the proposed adversarially trained ensemble achieves the strongest overall trade-off between classification performance on the robustness evaluation set and evasion resistance, reducing the average ASR from 0.41 to 0.03 and achieving an RI of 0.98.