Evaluating the Robustness of Machine Learning-Based Network Intrusion Detection Against Feature-Space Adversarial Evasion: An InSDN Study

Abstract

Machine learning-based network intrusion detection systems often perform extremely well on standard test datasets. However, high accuracy on clean data does not necessarily mean that these models can withstand carefully designed input manipulation. In this study, we examine the robustness of three widely used classifiers, namely Random Forest, XGBoost, and a multilayer perceptron (MLP), using the InSDN dataset. The problem is treated as a binary classification task in which network traffic is categorized as either Normal or Attack. Before training the models, duplicate records were removed, while identifier and timestamp fields were excluded from the dataset. The remaining data was processed using training-only preprocessing, and the 40 features with the highest mutual information scores were selected for the experiments. The three classifiers were initially tested using an untouched test set to establish their performance under normal conditions. They were then evaluated against a controlled feature-space adversarial evasion procedure, where bounded directional perturbations were applied using normalized budgets of ε ∈ {0.01, 0.025, 0.05, 0.10}. The models achieved almost perfect results on the clean test set, with F1 scores of 99.9927% for Random Forest, 99.9952% for XGBoost, and 99.9855% for MLP. However, their performance changed considerably when adversarial perturbations were introduced. For example, with a positive perturbation direction at ε=0.01, Random Forest recorded an attack success rate (ASR) of 94.09%. XGBoost reached an ASR of 93.15% at ε=0.05, while MLP reached 87.78% at the same perturbation budget. At ε=0.10 in the positive direction, both Random Forest and MLP reached an ASR of 100%.