Comparative Performance Analysis of Machine Learning Models for Binary Classification of Fileless Malwares Using Volatile Memory Forensics

Abstract

Fileless Malware is one of the fastest-evolving and hardest-to-detect types of malware. The malware resides solely within System Memory and never uses typical files as its payload. Fileless malware can utilize legitimate software like PowerShell and WMI, along with LOLBins to remain completely hidden in System Memory, eliminating any digital footprint on the physical hard drive, thus allowing it to elude detection by Signature-based Antivirus systems. In this paper we present a Hybrid Detection Framework that utilizes both Machine Learning and Memory Forensics for detecting malicious activity that operates in a file-less manner. Volatility extracts relevant data from the memory of an active system, creating structured dataset(s) containing Process Behavior, DLL injection Activity, Network Indicators and Script Execution Tracing. Multiple Machine Learning Models have been trained, including Random Forest and XG Boost to discern malicious from benign behavior. Our proposed Hybrid Model provides significant improvements over existing methods for file-less malware detection while reducing False Positives.