Understanding Human Motion from Depth Sensors: Activity Recognition and Age Group Recognition Using Skeleton Data

Abstract

Human Activity Recognition (HAR) plays a significant role in various applications, from learning a discipline to physical rehabilitation. In older adults, activity patterns can indicate levels of frailty, which helps inform the design of physical training programs to prevent falls and maintain mobility. HAR sensing ranges from wearable sensors such as IMUs and RGB cameras to video, specialized gait laboratories, perturbation units, VR, and other modalities. This paper presents a comprehensive study of depth-based, skeleton-driven HAR and age group recognition (AGR) using data collected from real-world nursing home environments. Depth sensors offer a privacy-preserving and non-invasive alternative to wearable and RGB-based systems, enabling continuous 24-h monitoring without requiring user compliance. We systematically evaluate multiple modeling paradigms, including classical machine learning models (DT, RF, KNN, SVM, HMM, HMM+SVM), sequence-based models (LSTM, TCN, ARNN), and graph-based approaches, using skeletal joint data extracted from depth images. Experiments are conducted on two heterogeneous datasets: NTU RGB+D (younger adults) and ETAP-DID (older adults). We analyze the impact of different joint subset configurations (full-body, limb-only, leg-only, and torso-only) and compare raw joint representations with handcrafted time-series features (TSFEL) for frame-based HAR. Beyond activity recognition, we introduce an AGR pipeline to distinguish younger from older adults based on skeletal motion patterns. We investigate multiple feature representations, including absolute joint positions, root-relative coordinates, bone vectors, and joint velocities, and provide interpretability through feature importance and saliency analysis to identify age-discriminative joints and motion cues. Our study provides a comprehensive analysis of various HAR models applied to depth data, examining model performance and the contribution of joint-based features to HAR and AGR. Our study highlights the potential for personalized privacy-preserved monitoring and intervention in nursing homes.