Skip to content
Book Open access

RemoPhysEmo: A Multimodal Emotion Dataset with Remotely Sensed Physiological Signals via Video and Radar

Nov 2026 · 0 citations

Abstract

Emotion recognition is a fundamental task in affective computing, and most current works consider multimodal fusion to integrate comprehensive information for better performance. Among the modalities, physiological signals present a specialty, as on the one side, they are hard to alter, providing more objective evidence for emotion understanding; on the other side, they require delicate equipment to be attached to the body, which is inconvenient and interfering for the monitored persons. Existing multimodal emotion research concerns more of non-physiological modalities, such as video, audio, and text, whereas physiological signals remain insufficiently explored. To bridge this gap, we introduce a new multimodal emotion dataset with remotely sensed physiological signals by millimeter-wave radar and facial videos. The new dataset is targeted to facilitate two research fields: 1) multimodal remote physiological signal measurement, and 2) physiological-behavioral fusion for emotion recognition with only remote sensors. Furthermore, beyond clip-level emotion annotations, we also provide fine-grained intensity labels, enabling more detailed temporal emotion analysis. Extensive benchmarks were established on the new dataset, including uni- and multimodal approaches for remote physiological signal measurement and emotion recognition. It is expected that the dataset and benchmarks provide a new testbed and open up new possibilities for future research. The dataset is available at: https://github.com/helloworld-YY/RemoPhysEmo.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.