Representation-Aware Knowledge Distillation for Efficient Malware Family Classification

Abstract

Malware family classification is an important cybersecurity task, but designing models that are both accurate and compact remains challenging due to malware obfuscation and computational constraints. This paper investigates efficient malware image classification by jointly considering byte-to-image representation, teacher model selection, and knowledge distillation. Using the Microsoft BIG 2015 dataset, raw byte files are transformed into three visual representations: grayscale images, Raw-Entropy-Missingness Visualization (REM-Vis), and frequency-based maps. For each representation, candidate convolutional backbones are screened on the validation set to select a suitable teacher, while a common lightweight student network is trained using hard-label supervision and teacher-guided distillation. Experimental results show that representation choice strongly affects downstream performance. Under the validation-selected best-pipeline protocol, grayscale images achieve the best student performance, with 98.94% accuracy, 97.73% Macro-F1, and 97.24% balanced accuracy. REM-Vis remains competitive, reaching 98.50% accuracy and 97.33% Macro-F1, whereas frequency-based maps obtain 97.79% accuracy and 96.44% Macro-F1. In the REM-Vis setting, feature-level distillation improves the compact student from 96.69% to 97.33% Macro-F1 while maintaining only 0.61M parameters. EfficientNet-B0 is selected as the teacher for all three representations, and feature-level distillation provides the best distillation setting. These results indicate that visualization strategy, teacher selection, and distillation objective should be optimized jointly for efficient malware family classification.