TL;DR
Cross-Modal Alignment Regularization (CMAR) is proposed, a method designed to improve LLMs by aligning their internal representations with those of vision models during training by bringing the internal representations of the language and vision models closer together.
For the coarse attributes the authors study, MLLMs encode the visual evidence but cannot reliably control their reliance on it, indicating that for the coarse attributes they study, MLLMs cannot reliably control their reliance on it.
An evidence-driven multimodal reasoning framework that utilizes a Bloom-inspired taxonomy as a hierarchical reasoning protocol and quantitatively analyzes the trace to make evidence usage and reasoning progression explicit is proposed.
Evaluating representative proprietary and open-source multimodal models, it is found that visual reasoning is strongly model- and environment-dependent, with no single setting consistently dominating across tasks.
This work investigates whether VLMs can be trained to reason about AI-generated image edits using reinforcement learning (RL) rather than explicit reasoning supervision, and introduces effective intersection over union (eff-IoU), a unified metric to jointly evaluate detection and localization.
Modality-Informed Reciprocal Reasoning Optimization (MIRROR), a reinforcement learning approach for improving multimodal reasoning via self supervision, is developed and improves over standard RL and yields more accurate and consistent behavior across modalities.
Modality reliance in VLMs is not fixed, but varies across tasks, evidence structures, models, and evaluation settings, which shows that modality reliance in VLMs is not fixed, but varies across tasks, evidence structures, and evaluation settings.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
