Facial Expression Recognition using Label-and Scale-Distribution-aware Loss-assisted Cross Vision Transformer

Abstract

In recent years, Facial Expression Recognition (FER) has gained attention as a crucial area of computer vision. However, FER methods depend on single-source facial data that often experience reduced accuracy and instability because of challenges such as facial occlusion and variations in illumination. Moreover, existing FER approaches fail to understand the salient facial regions, resulting in poor interpretability and performance. Hence, this research proposes the Label- and Scale- Distribution-aware Loss-assisted Cross Vision Transformer (LSDL-CrossViT) to solve the issues in FER. The proposed model captures the global facial structure and local expression-specific cues through cross-attention-based feature learning. By incorporating the attention mechanism through discriminative loss design, the pipeline improves class separability and reduces classification errors across multiple facial expression categories. This research considers three benchmarks such as RAF-DB, AffectNet-8, and CK+, to estimate the complete performance of the proposed approach. The experimental results illustrate that the proposed LSDL-CrossViT gains a balanced accuracy of 93.18%, 71.84%, and 99.01% on the RAF-DB, AffectNet-8, and CK+ datasets, respectively, as compared to the reproduced Distract your Attention Network (DAN) and Self Learning Weight Network related to Label Distribution Training (SLW-LDT) models.