PIF-Detector: An Accurate Low-Overhead Method for Detecting Persistent and Infrequent Flows

Abstract

Persistent and infrequent (PI) flows refer to packet flows that last for a long time at low packet arrival rates in network traffic. It is crucial to accurately find these flows for uncovering stealthy network threats, such as advanced persistent threats (APTs) and low-rate denial-of-service attacks (LDoS). To this end, we propose a low-overhead and accurate method for detecting persistent and infrequent flows called PIF-Detector, which separates the monitoring and identification functions of PI flows. Specifically, we design a monitoring table composed of bucket arrays to store and monitor potential PI flows, and an identifier composed of compact bit vectors to recognize potential PI flows. For the monitoring table, we design an adjacent-bucket relocation strategy to improve bucket utilization and thereby enhance PI-flow detection accuracy. For the identifier, we track flow arrivals across consecutive time windows using compact bit vectors to detect PI flows with low memory overhead. Finally, we perform theoretical derivations on the false positive rate and the false negative rate of PIF-Detector, and evaluate its performance through experiments on three datasets. Experimental results show that PIF-Detector achieves up to 99.57% precision and 99.93% recall, outperforming state-of-the-art solutions under the same memory overhead.