Empirical Evaluation and New Insights of Concept Drift in ML-Based Android Malware Detection

Abstract

Despite outstanding results, machine learning-based Android malware detection models struggle with concept drift, where rapidly evolving malware characteristics degrade model effectiveness. This study examines the impact of concept drift on Android malware detection, evaluating two datasets and nine machine learning and deep learning algorithms, as well as Large Language Models (LLMs). Various feature types—static, dynamic, hybrid, textual, and image-based—were considered. The results showed that concept drift is widespread and significantly affects model performance. Factors influencing the drift include feature types, data environments, and detection methods. Balancing algorithms help with class imbalance but do not fully address drift, which primarily stems from the dynamic nature of malware. No strong link was found between the type of algorithm used and concept drift; the impact was relatively minor compared to other variables because hyperparameters were not fine-tuned, and the default algorithm configurations were used. The LLM evaluation is treated as an exploratory baseline because the original feature spaces were compressed using PCA to satisfy token-length constraints. Under this setting, LLMs showed promising few-shot performance but remained sensitive to temporal drift.