Multi-Dataset Benchmarking of TF-IDF-Based Supervised Machine Learning Algorithms for Phishing and Spam Email Detection

Abstract

Email phishing remains one of the most persistent cybersecurity threats, exploiting users through deceptive emails to obtain sensitive information and compromise digital systems. As phishing attacks continue to evolve in complexity, conventional rule-based detection techniques have become less effective, creating a need for intelligent machine learning-based approaches. This study presents a multi-dataset benchmarking of five supervised machine learning algorithms: Logistic Regression, Random Forest, Decision Tree, Linear Support Vector Classifier (Linear SVC), and Extreme Gradient Boosting (XGBoost) for phishing and spam email detection. The models were evaluated on five publicly available benchmark datasets (CEAS_08, Phishing_Email, Ling, Enron, and Spam) using a unified, leakage-safe pipeline based on text cleaning, binary label mapping, and TF-IDF feature extraction (maximum 300 features). Model performance was assessed with Accuracy, Precision, Recall, F1-score, AUC, Precision-Recall AUC (PR-AUC), Matthews Correlation Coefficient (MCC), and Balanced Accuracy via five-fold stratified cross-validation, with all preprocessing steps performed strictly inside training folds and an untouched 20 % hold-out set reserved before any model selection. Datasets larger than 25 000 messages were stratified and subsampled to 20 000 messages to accommodate memory constraints while preserving class proportions. Experimental results show that Random Forest achieved the highest mean F1-score (0.9353) across the five datasets, followed by Logistic Regression (0.9261), Linear SVC (0.9180), XGBoost (0.9077), and Decision Tree (0.8809). A Friedman test confirmed a statistically significant difference among the algorithms (χ² = 15.84, p = 0.0032). The findings provide a fully reproducible classical machine-learning baseline and highlight the importance of leakage-safe evaluation protocols when comparing algorithms for email phishing and spam detection.