Machine Learning and Deep Learning Approaches for Anomaly Detection: A Review

Abstract

Anomaly detection is a crucial machine learning problem designed to uncover observations or patterns that are unusual, or outlier behaviors. It is commonly used in cyber security, Internet of Things (IoT), industrial monitoring, health care and other data-driven systems. But anomaly detection is still a challenge due to the rarity, labelability, heterogeneity, and fluctuation of data distributions. In response to these challenges, a range of machine learning and deep learning techniques have emerged in recent years, including distance-based methods, classification models, clustering, reconstruction-based learning, sequential modelling, and sophisticated hybrid architectures. This review focuses on the key methods in the fields of ML and DL for anomaly detection, highlighting their mathematical formulation, principles, anomaly scoring methods, and applicable limitations. The studies reviewed show that DL approaches are suitable for complex non-linear and temporal patterns while ML approaches are suitable for structured and resource limited applications. The review also points out that class imbalance, false positives, generalisation, interpretability and computational requirements are challenges.