An Explainable Hybrid Transformer-XGBoost Framework for Cyber Threat Detection

Abstract

The growing complexity and frequency of cyber-attacks against today's digital infrastructure are creating cyber threat detection and response challenges. In this paper, two Machine Learning (ML) and Natural Language Processing (NLP) methods based on Explainable Artificial Intelligence (XAI) for intelligent cyber threat detection and response are presented and compared on the basis of the SOREL-20M dataset. The model XAI-Transformer-XGBoost was developed and tested in MATLAB Simulink and Python TensorFlow using a series of simulations to incorporate a deep contextual threat representation with explainable ensemble classification. The proposed hybrid model was evaluated with CNN-LSTM, Feature Selection and Ensemble Generative Model (FSEGM), and standalone XGBoost models in terms of Accuracy, Sensitivity, Specificity, Training Time, and Cohen’s Kappa Score. Experimental results showed that the proposed XAI-Transformer-XGBoost model outperformed CNN-LSTM (96.41% accuracy), FSEGM (97.83% accuracy), and XGBoost (98.26% accuracy) with an accuracy of 99.12%, a sensitivity of 98.87%, a specificity of 99.34%, a Cohen’s Kappa of 0.982 and a training time of 118 seconds. The proposed approach showed improvements of 2.81%, 1.31%, and 0.87% in accuracy; 3.45%, 1.84%, and 1.12% in sensitivity; 2.96%, 1.52%, and 0.95% in specificity; and 4.91%, 2.71%, and 1.66% in Cohen’s Kappa score over CNN-LSTM, FSEGM, and XGBoost, respectively, while reducing training time by 21.9%, 14.5%, and 8.4%. The results show that providing explainable AI alongside hybrid Transformer and gradient-boosting methods is an effective approach that can enhance cyber threat intelligence, increase transparency of AI models, and facilitate faster and more dependable automated security response systems in practical cybersecurity applications.