Resilient Semantic Threat Detection at the Edge: A Knowledge Distillation Framework for SMS Spam Classification

Abstract

The ubiquity of Application-to-Person (A2P) messaging has inadvertently created a robust vector for mobile security threats, specifically Smishing (SMS Phishing). Traditional lexical filters, such as Naive Bayes and Support Vector Machines (SVM), exhibit diminishing returns against modern obfuscated attacks due to their inability to interpret semantic context. While Transformer-based architectures like BERT have revolutionized Natural Language Processing (NLP), their computational latency renders them impractical for real-time deployment on resource-constrained mobile edge devices. This study proposes a high-efficiency detection framework utilizing DistilBERT, a distilled knowledge representation of the BERT transformer. By fine-tuning this architecture on the UCI SMS Spam Collection, we achieved a testing accuracy of 99.04% and a weighted F1-score of 0.9904. Notably, the model maintained a precision of 0.97 on the minority spam class, effectively mitigating the class imbalance problem without synthetic data augmentation. These results substantiate the viability of Knowledge Distillation as a mechanism to deploy state-of-the-art semantic security filters on edge infrastructure.     Keywords - SMS Spam, Knowledge Distillation, DistilBERT, Cybersecurity, Edge AI, NLP, Smishing, Transfer Learning