Xiao-Ting Wu

Learning Multimodal Sensor Fusion with JEPA Representations for Breathing Activity and Pose Recognition

Predictive video representations do not by themselves specify how a multimodal system should handle missing sensors. We transfer frozen V-JEPA2 features to a set of heterogeneous sensors comprised by RGB video, depth video, and radar time series. A shared fusion model uses modality queries, fixed sensor slots, and expl...