Driver Intention Recognition Under Autonomous Takeover Scenarios Based on EEG and Eye-Tracking Fusion

Abstract

To address the challenges of recognition lag and limited representational capacity of single-modal physiological information in driver intention recognition under human–machine co-driving environments, this study constructed a highway driving takeover experimental scenario. Typical takeover tasks including obstacle avoidance takeover, adjacent vehicle cut-in takeover, and curved section takeover were designed, with simultaneous collection of drivers’ electroencephalography (EEG) and eye-tracking data. On this basis, a multi-modal physiological information fusion method for driving intention recognition was proposed. First, areas of interest (AOIs) were defined to extract visual behavioral features, including fixation duration, fixation count, and pupil diameter, while eye-tracking recurrence plots were generated based on phase space reconstruction. Meanwhile, event-related potential (ERP) analysis was performed on the EEG data to extract N100, N200, and P300 components, and continuous wavelet transform was applied to construct EEG time–frequency maps. Subsequently, the EEG time–frequency maps and eye-tracking recurrence plots served as the dual inputs to the dual-branch Squeeze-and-Excitation Convolutional Neural Network (SE-CNN) model, which was further enhanced by fusing ERP features and eye-tracking statistical features, ultimately achieving accurate recognition of four types of driving intentions. The model results demonstrated that the dual-branch SE-CNN model achieved a recognition accuracy of 88.29% on the four-class intention recognition task, with precision, recall, and F1-score reaching 88.78%, 88.26%, and 88.31%, respectively—significantly outperforming both single-modal models and the dual-branch Convolutional Neural Network (CNN) model without the SE attention mechanism. The proposed model can effectively integrate EEG neural activity signals and eye-tracking visual behavioral information, thereby substantially enhancing driving intention recognition performance. This provides theoretical foundations and technical support for applications such as early driver intention perception, intelligent takeover decision-making, and human–machine collaborative control in human–machine co-driving environments.