A multi-backbone framework with architecture-adaptive pruning and gated feature fusion for driver behavior recognition

Abstract

Driver behavior recognition (DBR) is an important component of intelligent driving systems because it enables continuous monitoring of unsafe driver actions in complex environments. However, multi-backbone DBR models often provide improved representation capability at the cost of high computational complexity, while compression may introduce feature imbalance and unstable optimization. To address these issues, this study proposes a Multi-Backbone Adaptive Pruning and Fusion (MBAPF) framework. A Backbone-Aware Adaptive Pruning (BAAP) method compresses backbones using architecture-specific pruning granularities, where “adaptive” refers to adaptation to backbone structures rather than automatically learned pruning ratios. A Gated Multi-Backbone Feature Fusion (G-MBFF) mechanism then performs sample-dependent calibration of the compressed backbone features, and a Progressive Fine-tuning Strategy (PFS) stabilizes joint optimization by gradually expanding the trainable parameter scope. On the driver-independent SAA13 test set, MBAPF achieves 92.61% accuracy, 93.18% precision, 92.72% recall, and a 92.95% F1-score. Compared with the original multi-backbone fusion model, MBAPF reduces the parameter count from 74.334 to 54.312 M and GFLOPs from 12.873 to 8.279 G, while achieving a measured throughput of 27.5 FPS. Leave-one-dataset-out experiments further evaluate generalization to unseen acquisition domains. These results demonstrate that MBAPF provides a favorable trade-off among recognition performance, model complexity, and inference efficiency under the evaluated desktop GPU environment.