Dependency, Compression, and Synergy: A Unified Information-Theoretic View of Multimodal Learning
This survey presents an information-theoretic perspective connecting Mutual Information, the Information Bottleneck, and Partial Information Decomposition as progressively refined views of multimodal information processing, and discusses how emerging multimodal foundation models instantiate these principles at scale.