Explaining the Black Box: An XAI-Driven Trustworthiness Audit of ML-Based IoT Intrusion Detection Across Attack Categories

Abstract

Machine learning classifiers for Internet of Things (IoT) intrusion detection routinely report accuracy above 99%, yet this conceals systematic failures on operationally important minority categories such as BruteForce and Web-based attacks, and the global explainability analyses usually applied to these models say nothing about how their decision logic varies from one threat type to another. We argue that a third property matters alongside detection and explanation: whether a classifier explains different attacks in a consistent way. To measure it, we introduce the Explanation Consistency Score (ECS), a model-agnostic metric based on the Jaccard similarity of top-K SHAP (SHapley Additive exPlanations) feature sets. To the best of our knowledge, no directly equivalent measure was identified in the literature reviewed here for cross-category explanation consistency in multiclass intrusion detection. Applying ECS in a per-category audit of four classifiers (Decision Tree, Random Forest, XGBoost, and Logistic Regression) on CICIoT2023, we find that performance and explanation consistency pull against one another. XGBoost is the strongest detector (Macro-F $1=0.883$ , Balanced Accuracy = 0.847), with near-perfect recall on Distributed Denial of Service (DDoS), Denial of Service (DoS), and Mirai but F1 of only 0.708 and 0.706 on BruteForce and Web-based attacks, and among the tree-based models it records the lowest ECS, 0.396 at K = 10, against 0.521 for Decision Tree. Logistic Regression records a lower ECS still, 0.291, but performs poorly on detection and so falls outside the pattern. A within-model correlation between ECS and per-category F1 is reported for completeness in Section V-H, but with only seven categories it is underpowered and we do not draw conclusions from it. We treat this trade-off as an empirical model-selection characteristic to be audited, not a defect to be removed.