HYBRID DEEP LEARNING MODEL FOR EMOTION RECOGNITION SYSTEM WITH PROPOSED WDCRF TECHNIQUE

Abstract

Facial Emotion Recognition (FER) is widely used for many applications and Deep Learning (DL) has significantly advanced performance of FER, yet many models fail to capture temporal coherence and realistic emotional transitions in visual sequences. This study introduces a hybrid of Residual Network 18-Layer (ResNet-18), Bidirectional Long Short Term Memory (BiLSTM) and proposed Weighted Directional Centroid Ranking Function (WDCRF) framework that integrates spatial feature extraction, sequential dependency modelling, and psychologically guided temporal smoothing within a unified end-to-end architecture. The proposed WDCRF is a hand-crafted feature alignment and semantic grounding module that bridges the gap between traditional descriptor-based features (LBP, HOG, Gabor) and deep neural network representations. Its job is to ensure that what the deep model learns stays anchored to psychologically validated, human-interpretable facial cues preventing the deep features from drifting toward patterns that are statistically convenient but emotionally meaningless and integrated in hybrid of ResNet-18 and BiLSTM models to improve overall performance. An improved pre-processing pipeline enhances FER2013 images through alignment, mask-aware augmentation, and adaptive histogram equalization to preserve subtle micro-expressions. Experimental results show that the proposed hybrid model i.e. ResNet-18+BiLSTM+WDCRF achieves 88.85% accuracy, 87.0% precision, 87.0% sensitivity, 97.3% specificity and F1-score of 87.0%, across all emotion classes followed by hybrid Resnet-18 and BiLSTM, ResNet-18 only and BiLSTM only. Models were also tested on different types of occlusion and results were found satisfactory. During ablation study also hybrid of ResNet-18, BiLSTM and WDCRF has shown significant performance with F1-Score of 66%.