Adaptive Deep Learning Approach for Detecting Malware in Encrypted Network Traffic
Abstract
The increasing use of encryption in modern communication networks creates a challenge for malware detection because malicious activity can be concealed within encrypted traffic. Conventional signature matching, deep packet inspection, and payload-based analysis become less effective when packet contents cannot be directly examined. This paper presents a deep learning-based framework for detecting malware-related encrypted traffic using flow-level metadata and statistical communication behavior rather than packet payloads. The framework extracts traffic characteristics such as packet count, flow duration, packet-size statistics, transmission rate, and inter-arrival behavior. A preprocessing stage removes duplicate and incomplete records and normalizes the extracted features. Behavioral profiling is then used to represent communication patterns, followed by adaptive feature weighting and deep learning-based classification. A risk assessment stage combines the learned classification output with behavioral information to support the final benign or malicious decision. The reported evaluation uses accuracy, precision, recall, and F1-score to assess detection performance. Based on the confusion matrix reported in this manuscript, the framework achieves 98.12% accuracy, 97.83% precision, 98.38% recall, and 98.10% F1-score. The approach avoids payload decryption and therefore provides a privacy-preserving direction for malware detection in encrypted enterprise, cloud, and IoT traffic.