AI-based vision techniques for human activity recognition in surveillance videos
Human Activity Recognition (HAR) is increasingly being incorporated in intelligent surveillance systems; however, the majority of the current techniques do not perform well in real-world situations that incorporate live video streams, dynamic backgrounds and complex human behaviours. In particular, there is a significant difference in comparing HAR algorithms when they operate on trimmed (segmented) video vs. untrimmed (streaming) video. This article provides an overview of vision-based HAR systems designed for surveillance applications. The article compares the performance of traditional machine learning techniques with recent deep learning architectures such as CNNs, RNNs, TCNs, and Transformers, based on accuracy, computational cost, and suitability for real-world disorderly plotting. The article also examines how well publication datasets model video surveillance scenarios and describes the limitations of each dataset. The article identifies several practical issues associated with each technique and dataset (occlusions, illumination changes, camera motions, and crowded environments). This article also highlights the major contributions regarding the trimmed and untrimmed video-based HAR methodologies and provides an exhaustive review of models, datasets, and practical issues related to their real-world application within the context of surveillance. Finally, the authors also provide guidance on potential future directions of research, such as Edge AI, product data integration & explainable AI, and techniques focused on ensuring privacy.