PLGSA-Transformer: Periocular Landmark-Guided Attention with Occlusion-Adaptive Cosine Thresholding for Cross-Modal Masked and Unmasked Face Recognition
Results confirm that encoding periocular geometry into attention, with Transformer modelling and occlusion-adaptive thresholds, yields a robust, scalable solution for cross-modal masked face recognition.
Abstract
The widespread adoption of facial masks, accelerated by COVID-19 and mandated in security-sensitive settings, has exposed limitations of conventional face recognition systems. Existing approaches relying on fixed cosine thresholds, non-adaptive CNNs, and purely data-driven features fail to generalize when facial regions are occluded, creating a gap between lab performance and real-world deployability. This paper proposes PLGSA-Transformer, a cross-modal face matching framework with three contributions. First, Periocular Landmark-Guided Spatial Attention (PLGSA) uses MediaPipe landmarks to compute Gaussian heatmaps over the eye, brow, and forehead regions, fusing them with EfficientNetB3 features via a learnable residual gate to direct attention toward discriminative visible regions. Second, a Hybrid CNN-Transformer Branch reshapes feature maps into tokens processed by a two-layer Multi-Head Self-Attention encoder, enabling cross-regional dependency modelling. Third, the Occlusion-Adaptive Cosine Threshold (OACT) is a jointly trained head that raises the matching threshold in proportion to predicted occlusion severity. The model is evaluated on 858 images from Zenodo MDMFR (60%), Kaggle CelebA-HQ masked collection (25%), and author-collected images (15%), spanning both genders, ages 21-75, with varied mask types, trained via a unified loss combining contrastive verification, identity classification, and occlusion cross-entropy. PLGSA-Transformer achieves 97.22% pair verification accuracy with ROC AUC 1.0000, surpassing VGG-16-based MUFM (Abdullah et al., 2025; 95.0%), HOG classifiers (Adnan et al., 2020; 85.0%), and Feature-based Structural Measure (Shnain et al., 2017; 86.61%). These results confirm that encoding periocular geometry into attention, with Transformer modelling and occlusion-adaptive thresholds, yields a robust, scalable solution for cross-modal masked face recognition.
Facial occlusion degrades face recognition by creating scale-inconsistent identity cues across visible regions and amplifying responses to irrelevant occluders. To address these coupled problems, this study proposes a pyramid-guided multi-scale attention framework based on scale alignment and reliability-aware feature refinement. Hierarchical features are first projected into a shared semantic space, after which local, intermediate, and global dependencies are adaptively weighted according to the available facial information. Channel–spatial refinement is then used to suppress unreliable responses from occluded regions, while an angular-margin objective preserves inter-identity separability from incomplete facial evidence. Experiments on CASIA-WebFace and occluded LFW show that the proposed method achieves an accuracy of 99.26% under clean conditions and 82.72% under occlusion, with an ROC-AUC of 0.8351. At 80% occlusion, the proposed method outperformed the strongest recent baseline, HMPA-GFAF, by 1.41% points and the direct Inception-ResNet-v1 + ArcFace baseline by 5.88% point. These results demonstrate that the proposed framework improves occlusion robustness without sacrificing clean-face recognition performance, indicating its practical potential for identity verification and access-control applications involving masks, glasses, and other partial facial occlusions.
Child face identification from surveillance video remains difficult because facial crops are frequently low-resolution, blurred, partially occluded, and captured under unstable illumination. Age-related facial variation further increases the difficulty of maintaining discriminative identity embeddings for children. This paper presents RCF-Net, a degradation-aware hybrid CNN–Transformer architecture that combines surveillance-oriented image degradation, dual-branch local/global feature extraction, and learnable cross-attention fusion. MTCNN is used for face detection and alignment, ArcFace supervision is used for discriminative embedding learning, and DeepSORT can optionally be integrated to improve temporal identity consistency in video streams. To address deployment concerns raised by surveillance use, the revised framework also specifies age-progression handling, latency-aware scheduling for live video, multi-camera scaling, and adversarial/spoof-risk safeguards. Experiments are conducted using public face datasets, namely VGGFace2, CASIA-WebFace, CelebA, and IMDB-WIKI, with child-oriented filtering and synthetic surveillance degradations. Compared with representative CNN, transformer, and hybrid baselines, RCF-Net achieves the best overall accuracy of 91.4% and yields the strongest robustness under low-resolution, blur, and occlusion stress tests. The results indicate that explicit degradation modeling and local-global feature fusion are complementary for surveillance-oriented child face identification.
R. Arora, Akash Pandey, Navjeet Kaur· 2026 4th International Confe...· 0 citations
Recognizing facial emotions automatically from images/videos (FER) still represents a difficult problem for emotion computing, mainly due to variations in the face pose, lighting, occlusion, facial features, and expression intensity in the wild. Recent CNN–Transformer-based hybrid models like POSTER have leveraged local feature learning, landmark guidance, and global dependency modeling to achieve strong performance. Yet these methods give the main focus to spatial and contextual representations while not really going deep into adaptive channel-wise feature importance over multi-scale representations. As different feature channels represent emotions in varying degrees, it is likely that by treating all feature channels equally, one would limit the ability of the learned features to discriminate effectively. To overcome this weakness, this article presents a ResNet-18–Transformer landmark-guided module called SE-POSTER that fuses lightweight Squeeze-and-Excitation (SE) attention modules into the multi-scale feature pyramid of the baseline POSTER architecture. The proposed method carries out feature channel recalibration adaptively at the level of features before Transformer-based global attention modeling, thus allowing the network to focus on emotionally informative feature channels and suppress less relevant responses. The inclusion of SE attention in the network enhances fine, mid, and global levels of feature representations at a very low cost in terms of computation. On the basis of the RAF-DB, FERPlus, and AffectNet datasets, enormous experiments prove that the SE-POSTER framework proposed is capable of steadily boosting recognition accuracy relative to the baseline POSTER and several state-of-the-art FER methods. Especially, the proposed model delivers 92.78% accuracy on RAF-DB while it also shows better robustness and generalization capability under difficult real-world conditions. Moreover, additional ablation studies reveal that multi-level channel recalibration is effective in improving discriminative emotional feature learning.
Alpamis Kutlimuratov, K. Sharipov, Piratdin Allayarov et al.· Informatics· 0 citations
Facial identity identification in unrestricted real-world environments may benefit from this model, which performs well in identifying and verifying low-quality and cross-pose masked faces and outperforming the various state-of-the-art methods and previously proposed methods.
P. Kaur, Taqdir Kaur, Sahezpreet Singh· Engineering Research Express· 0 citations
Face recognition systems applied to smart surveillance settings often experience poor performance when the faces are partially occluded by a mask or other objects. Occlusions eliminate critical facial information, which makes face identification much more difficult for traditional deep learning models. To solve this issue, a hybrid deep learning model utilizing convolutional neural networks and transformer-based attention mechanism is proposed in this study for robust masked and occluded face recognition. The framework uses the ResNet50 backbone for obtaining the discriminative local facial favorable features, and the Vision Transformer module for obtaining long-range context relationships between facial regions. In addition, an Adaptive Occlusion Attention Module is introduced to Zurcrook visible facial areas and neglect the corrupted features to occlusions. Experiments were carried out on the Real-World Masked Face Dataset (RMFD) with 1205 images of 25 identities. The proposed model attained 93.46% training accuracy and Top-1 and Top-5 recognition accuracy were 51.87% and 81.33%, respectively. Additional occlusion experiments resulted in occlusion recognition accuracy of 28.63% and cross-dataset evaluation using MaskedFace-Net resulted in an average feature similarity of 0.8288. The results show that the proposed hybrid architecture enhances the recognition robustness of masked and partially obstructed facial images facing the surveillance situation.
R. R, Anbalagan E· 2026 4th International Confe...· 0 citations
Facial expression recognition technology is vital for security, verification, and personalization, but it faces challenges due to variations in scale, illumination, occlusion, and facial expressions. This paper presents a hybrid architecture that combines Vision Transformers (ViTs) to capture global context with EfficientNet-B3 for multi-scale feature extraction. Unlike simple concatenation, our approach projects the ViT’s [CLS] token and the EfficientNet’s global pooling features into a shared 512-dimensional space before merging, enabling better alignment of global and local features. When tested on the FERPlus dataset, it reaches an accuracy of 94.4 ± 0.3%, surpassing several recent methods, notably existing transformer- and CNN-based methods. Ablation studies show each component’s contribution, with the full model outperforming the no-fusion version by 2.6%. With around 98 million parameters and an inference time of ~23 ms per image, it balances efficiency and high performance, suitable for real-time use on suitable hardware. Evaluation via confusion matrix, t-SNE visualization, and comparisons with recent techniques such as HLA-ViT (90.13%), AU-ViT (90.15%), and CCFER (91.24%) demonstrates its robustness and discriminative feature learning. This work highlights the promise of hybrid deep learning architectures in tackling real-world facial expression recognition challenges.
Sasan Karamizadeh, Saman Shojae Chaeikar, Mazdak Zamani· Journal of Imaging· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
