SST-Net is proposed, a spatiotemporal collaborative attention enhancement framework for SLR that introduces input-dependent gates to dynamically fuse spatial and channel attention, rather than using fixed fusion weights.
Abstract
In complex environments, sign language recognition (SLR) is easily affected by background clutter, motion blur, and rapid movement. These factors can obscure subtle gesture patterns and weaken the discriminative spatiotemporal features required for recognition. To address these challenges, we propose SST-Net, a spatiotemporal collaborative attention enhancement framework for SLR. First, CGSA introduces input-dependent gates to dynamically fuse spatial and channel attention, rather than using fixed fusion weights. Second, the TFM and SFM both exploit inter-frame differences but use them differently: the TFM enhances motion-sensitive temporal features, whereas the SFM suppresses motion-dominated components to preserve spatial details; both are further refined through feature reweighting. Third, experiments on CSL-100, INCLUDE, and AUTSL show that SST-Net improves accuracy by 2–4 percentage points over strong baselines while using 35 M parameters and 115 GFLOPs, demonstrating a competitive balance between recognition performance and computational cost.
The proposed LCASP-MMANet framework provides an effective balance between accuracy, robustness, and computational efficiency for sign language recognition, and competitive results on the VOC benchmark demonstrate strong generalization capability.
Sign Language Recognition (SLR) has gained increasing attention, while Continuous Sign Language Recognition (CSLR) remains substantially more challenging than Isolated SLR (ISLR) due to the difficulty of accurately detecting sign boundaries within continuous signing streams. This challenge is intensified by co-articula...
R. Rastgoo, Kourosh Kiani, Sergio Escalera· Technologies· 0 citations
Isolated Sign Language Recognition (ISLR) requires an effective representation of both fine-grained hand articulations and the complex spatio-temporal motion patterns that define each sign. Existing approaches often rely solely on RGB sequences, optical flow, or skeletal keypoints, which limits their robustness under o...
S. Renjith, Aneesh Varghese, P. S.· IEEE Open Journal of the Com...· 0 citations
A self-attention-based VGG16 framework for static sign language gesture recognition of alphabetic and numeric hand signs and suggests that the proposed framework can serve as a promising building block for intelligent assistive technologies supporting inclusive educational environments.
Naoufal El-Marzouki, Imane Lasri, Fatim Ezzahrae Dorhmi et al.· International Journal of Onl...· 0 citations
To address the adverse effects of complex acquisition conditions on continuous sign language recognition (CSLR) performance and the increasing computational cost of standard self-attention with increasing video sequence length, this paper proposes a continuous sign language recognition method based on temporal–frequenc...
Xiang-Yang Sun, Chu-Han Wang, Zi-Han Cai et al.· Applied Sciences· 0 citations
Deep learning (DL) has shown significant potential in device-free gesture recognition (DFGR), enabling contactless device control and intuitive human–robot interaction. However, DL-based methods usually require a large number of labeled samples for training. Since the collection of annotations for radar signals is time...
Chao Fang, Yong Wang, Mu Zhou et al.· IEEE Transactions on Automat...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
