The Impact of Residual Connections on Facial Expression Recognition: An Ablation Study Based on the FER+ Dataset
TL;DR
It is suggested that, for a lightweight 18-layer network, residual connections contribute marginally to overall accuracy but help stabilize training and improve feature learning for underrepresented classes.
Abstract
Facial expression recognition, a key task in affective computing, has been applied in human– computer interaction, driver state monitoring, and psychological behavior analysis. Training deep networks often suffers from instability, and residual connections, which introduce skip connections across layers, help stabilize the training of deep models. To examine the specific role of residual connections, comparative experiments were conducted on the FER+ dataset using two 18-layer architectures: ResNet18 and Plain18. Plain18 shares the identical structure and training configuration as ResNet18 except that all residual connections are removed. Data augmentation, label smoothing, the Adam optimizer, and early stopping were consistently applied. On the test set, ResNet18 achieved an accuracy of 74.34%, while Plain18 reached 73.36%. Although the overall improvement is modest, the benefit of residual connections is more pronounced in certain categories. For example, the F1 score for the minority class “Contempt” increased from 0.28 to 0.39. Training curves also indicated that Plain18 exhibited signs of overfitting in later epochs, whereas ResNet18 maintained stable loss. These findings suggest that, for a lightweight 18-layer network, residual connections contribute marginally to overall accuracy but help stabilize training and improve feature learning for underrepresented classes. This work provides a practical reference for evaluating the effect of residual connections in facial expression recognition.