Multimodal models are expected to exploit synergistic information, that is, information available only when multiple modalities are observed together. In practice, however, a fusion module can fit the training objective by relying on a dominant modality or on information that modalities share, and the synergy between m...
Liang-Jian Wen, Qun Dai, Jiang Duan et al.· IEEE Transactions on Pattern...· 0 citations
Higher-order Representation and Information Learning (HRIL), which constructs an empirical cross-moment tensor over modality embeddings to represent multi-way interactions and employs Tucker decomposition to obtain a core tensor, complemented by a synergy-aware regularizer that prevents energy concentration and preserv...
Qun Dai, Liang-Jian Wen, Jiang Duan et al.· 0 citations
This survey presents an information-theoretic perspective connecting Mutual Information, the Information Bottleneck, and Partial Information Decomposition as progressively refined views of multimodal information processing, and discusses how emerging multimodal foundation models instantiate these principles at scale.
Liang-Jian Wen, Lin Li, Jiang Duan et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.