This work proposes Modality-Adaptive Decoding (MAD), a training-free method that adaptively weights modality-specific decoding branches based on task requirements based on task requirements, demonstrating that explicit modality awareness through self-assessment is crucial for robust multimodal reasoning.
Sangyun Chung, Se Yeon Kim, Youngchae Chee et al.· arXiv.org· 3 citations· ⚡2
This work presents an RS-specific formulation of the region selection paradigm, previously explored in natural-image MLLMs, and extends it to temporal change localization over multi-image sequences, and constructs a multi-task training and evaluation suite spanning localization, referring expression, visual grounding,...
J. Chung, Sungjune Park, Yeongyun Kim et al.· 0 citations
AV-STE is proposed, a modular streaming audio-visual front-end that restores corrupted semantic speech tokens from noisy audio and lip video before they reach the speech LLM, and remains entirely frozen, preserving its pretrained conversational capabilities.
This work decomposes video into view-invariant and view-variant latents and recompose them under language supervision encouraging clean decomposition of the two streams, and proposes PRISM, that achieves state-of-the-art results on EgoExo4D, EgoExoLearn, AE2, even surpassing in-domain models under zero-shot setting.
Youngchae Chee, Ho-Su Lee, Sungjune Park et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
