Abbas Edalat

Mitigating Strong-Modality Collapse in Multimodal Learning via Inverted Asymmetric Fusion

Fusing multiple modalities is expected to improve model performance. However, on the MultiHuSE dataset, early, late, and symmetric attention fusion often fail to outperform the best unimodal baseline (text). Pathway isolation of a symmetric attention fusion model reveals that the text-pathway accuracy drops from 74.9%...