Thermal-Guided Activity Gating for Multi-Modal Remote Physiological Measurement

Abstract

Remote photoplethysmography (rPPG) enables physiological signal estimation from facial video but remains sensitive to various factors such as illumination variation, motion artifacts, and spatial indiscrimination, where non-skin regions contribute equally to learned features despite carrying no pulsatile information. We propose a multi-modal rPPG estimation framework that integrates RGB and thermal modalities and introduces an unsupervised ActivityGate spatial gating module that constructs a subject-specific thermal activity map to suppress non-skin activations before the temporal transformer encoder. The proposed framework is further extended to simultaneous heart rate (HR) and breathing rate (BR) estimation via task-specific heads supervised by scalar frequency losses without requiring ground-truth waveforms. Evaluated on iBVP and SIM1 public datasets, the proposed model outperforms existing RGBT methods by 75.3% in MAE, with the ActivityGate alone attaining 71.9% MAE improvement on iBVP and 13.8% (HR) and 61.0% (BR) MAE improvements on SIM1 over the gate-free baseline at a negligible parameter overhead of $+0.25\%$ for RGBT fusion, of which the ActivityGate itself contributes only 0.0006%.