DRAPE (Dynamic Cross-Modal Prompt Generation), a prompt-learning framework that synthesizes continuous instance-specific soft prompts for MCIT, and achieves state-of-the-art performance among representative prompt-based and LoRA-based continual-learning baselines.
Multimodal continual instruction tuning (MCIT) aims to enable multimodal large language models to acquire new capabilities from sequential tasks while preserving previously learned knowledge. Existing methods primarily mitigate catastrophic forgetting by constraining parameter updates or separating task-specific adapta...
Tao Hu, Zhi-Nuo Zhou, Xia-Liang Tong et al.· 0 citations
Class-Incremental Learning (CIL) requires models to recognize new classes over time without forgetting previously learned ones. With the rise of vision-language pre-training, CLIP has become a strong foundation for CIL. A common design in CLIP-based CIL is to construct textual classifier weights by encoding class-name...
Tao Hu, Zheng-He Xie, Jing-Cai Guo et al.· 0 citations
This paper systematically identifies one of the fundamental challenges behind CIL, named feature collision, where the features learned by the current task-specific model may collide with those of the previous models, leading to forgetting of previously learned tasks and hindering the learning of new tasks.
Bowen Zheng, Zijun Shen, Da-Wei Zhou et al.· International Journal of Com...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.