Correlation-Based Leakage Screening is Insufficient for IoT/IIoT Intrusion-Detection Benchmarks: Evidence of Redundant Shortcut Encoding in Edge-IIoTset

Abstract

Machine-learning intrusion detection for the Internet of Things (IoT) and Industrial IoT (IIoT) routinely reports near-perfect accuracy on public benchmarks, yet the feature-level basis of that performance is rarely interrogated. This study benchmarks four supervised classifiers, random forest, gradient boosting, logistic regression, and a multilayer perceptron (MLP), on the Edge-IIoTset dataset and reports a result best read as a warning rather than a success. On the full feature set, an L2-regularized linear classifier attains perfect binary separation (accuracy 1.0000) of fourteen structurally heterogeneous attack types from benign traffic, an outcome that cannot plausibly reflect genuine attack behavior and instead indicates a label-encoding shortcut. A univariate correlation screen identifies four protocol-identity fields (dns.qry.name.len, mqtt.topic, mqtt.protoname, mqtt.conack.flags; $\vert $ r $\vert $ up to 0.966) as carrying it. Removing them degrades the linear model substantially (1.0000 to 0.9566) but leaves random forest classifying all but one of the 30,449 held-out records correctly (a single false positive; accuracy 1.0000) and the MLP at 0.9990. This collapse is a property of the features, not the encoding: repeating the linear model with one-hot encoding, and with all preprocessing fitted on the training partition only, reproduces it to four decimal places. Complementary detectors confirm the diagnosis, mutual information and recursive feature elimination flag shortcut-bearing features the linear screen misses, while adversarial validation (area under the curve 0.4959) locates the leakage in the features rather than the train/test split. A feature-ablation cascade shows accuracy declining only gradually, stabilizing near 0.84 as a conservative estimate of genuine signal. Correlation-based leakage screening is therefore necessary but not sufficient, and residual near-perfect performance should be treated as an unresolved red flag rather than as evidence of genuine non-linear attack signal. SHapley Additive exPlanations (SHAP) characterize the post-control decision basis (chiefly Hypertext Transfer Protocol (HTTP) request fields, destination port, and TCP flag and acknowledgment fields), but the released ML-ready file omits capture-session identifiers, so this study cannot adjudicate between genuine behavioral signal and residual session-structural leakage. Applied to CICIoT2023, the same protocol does not reproduce this signature ( $\vert $ r $\vert \le 0.53$ ), showing that it discriminates leakage-driven from signal-driven benchmarks. The author recommends dual (uncontrolled and controlled) reporting and session-aware grouped evaluation as minimum methodological practice for this and structurally similar multi-session IoT security benchmarks.