BPA-STGCN: Body-Part-Aware Spatio-Temporal Graph Convolutional Network for Stable Skeleton-Based Action Recognition

Abstract

Skeleton-based action recognition via graph convolutional networks (GCNs) has achieved remarkable progress, yet two persistent bottlenecks limit practical deployment: (1) systematic confusion among fine-grained actions that differ primarily in hand or finger movements, which the standard 25-joint skeleton cannot disambiguate; and (2) training instability under small batch sizes caused by BatchNorm (BN) running-statistics pollution, leading to catastrophic accuracy drops during training. In this paper, we propose BPA-STGCN (Body-Part-Aware STGCN), which addresses both challenges through an integrated framework of architectural and training innovations. First, a Partition Attention (PA) module adapting the Squeeze-and-Excitation concept to anatomically defined joint groups that decomposes the 25-joint skeleton into four anatomical partitions and learns sample-specific importance weights for each partition, enabling the model to focus on the most discriminative body region for each action. Then, the information losing global average pooling is replaced by a Temporal Pyramid Pooling (TPP) module adapting the temporal-segment and pyramid-pooling concepts to skeleton feature maps that captures multi-scale temporal dynamics through segmented pooling. Moreover, we design a stability-first training protocol comprising low-momentum BN, Mixup augmentation, gradient clipping, and extended warmup. The experiments are performed on the NTU RGB+D 60 and NTU RGB+D 120 dataset, and BPA-STGCN achieves 93.7% and 90.9% accuracy. Comprehensive ablation studies reveal that the architectural innovations and the stability protocol contribute complementarily, and that BPA-STGCN achieves the best accuracy–stability trade-off among all tested configurations.