Conference Open access

CoDA: Co-adaptive Dual-path Alignment for Vision-Language Models

Yi Zhang Rui Zhu Chan-Ni Li Xiao-Xu Li Zhan-Yu Ma Jing-Hao Xue

Sep 2026 · Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence · 0 citations · 36 references

TL;DR

In CoDA, a new adaptation framework that explicitly disentangles and coordinates cross-modal semantic alignment and intra-modal structural consistency is proposed, and it is shown that CoDA outperforms state-of-the-art parameter-efficient methods, particularly under few-shot learning and distribution-shift scenarios.

Abstract

Parameter-efficient adaptation methods, such as adapters and prompt learning, have become popular for transferring pre-trained vision-language models (VLMs) to downstream tasks. However, under limited supervision during adaptation, overly aggressive cross-modal alignment can distort the intrinsic structure of modality-specific representations, leading to degraded generalization. In this paper, we propose Co-adaptive Dual-path Alignment (CoDA), a new adaptation framework that explicitly disentangles and coordinates cross-modal semantic alignment and intra-modal structural consistency. In CoDA, we also propose a parent class labeling strategy that injects hierarchical semantic priors into textual prompts, further stabilizing alignment under limited supervision. Extensive experiments on eleven benchmark datasets show that CoDA outperforms state-of-the-art parameter-efficient methods, particularly under few-shot learning and distribution-shift scenarios. Our code is available at https://github.com/yizhang-ac/CoDA.

DOI Semantic Scholar

Read PDF

Similar papers

2026

Tiper: Tokenwise Prompt-Free Learning

Vision-language models (VLMs), such as contrastive language-image pre-training (CLIP), exhibit powerful zero-shot generalization capabilities. Parameter-efficient fine-tuning (PEFT) techniques, notably prompt learning, have been extensively explored to adapt these models to downstream tasks. However, their efficacy rem...

Tengfei Gong, Jun-Lin Wu, Yaxioong Chen et al. · IEEE Transactions on Geoscie... · 0 citations

Open access Jul 2026

Adaptive cross-modal alignment via symmetric prompt tuning for few-shot vision–language learning

This work proposes an Adaptive Cross-Modal Alignment via Symmetric Prompt Tuning for Few-Shot Vision–Language Learning (ACAS-PT) a unified framework that resolves query-side semantic blindness in vision-language few-shot learning via symmetric prompt tuning.

Feidu Akmel, Xun Gong, Abdalla Hadabi et al. · Journal of King Saud Univers... · 0 citations

#artificial intelligence Preprint Aug 2026

From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model

This work proposes a principled VLM TTA method called \algname, and theoretically reveals that the InfoNCE loss can be neatly reformulated as a Wasserstein OT formulation, thereby unifying the objectives of the inference and adaptation of VLMs to achieve their mutual benefits.

Qi Yu, Zhichen Zeng, Katherine Tieu et al. · 0 citations

Preprint Jul 2026

Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models

The results suggest that competitive MLLM can emerge from alignment alone, reducing multimodal extension to a lightweight projector-training problem that generalizes across modalities and adapts rapidly to each new LLM release.

Xuanru Zhou, Yiwen Shao, Jiahong Li et al. · 1 citation

Preprint Aug 2026

LLaVAFlow: Preserving Latent Alignment Flow for Parameter-Efficient Multimodal Fine-Tuning

This work argues that cross-modal alignment is implicitly captured in the information-compression trajectory, and proposes LLaVAFlow, an information-theoretic distillation framework that preserves alignment flow and enhances both downstream performance and generalization.

Mu-Yao Yuan, Mu-Yan Jiao, Jiang-Yong Ying et al. · 0 citations

Open access Aug 2026

RoFLIP: Robust and Fine-Grained Alignment for Vision-Language Compositional Reasoning

The Robust and Fine-grained training framework for CLIP-based vision-language models (RoFLIP) is proposed, enhancing both the robustness and granularity of vision-language alignment and underscore RoFLIP’s compositional reasoning and generalization abilities.

Yiwei Sun, Chuan-Bin Liu, Shan-Cheng Fang et al. · International Journal of Com... · 0 citations