C-GAP: Class-Aware and Online Prompting Improves Vision-Language Models on Imbalanced Classes

Visual Distribution Anchoring for Efficient Prompt Tuning

This work proposes VDA (Visual Distribution Anchoring), a training-free target adaptation framework that augments a frozen semantic classifier with class-level visual prototypes estimated offline from an unlabeled target pool, and shows that class-specific partitioning drives gains and that visually local pseudo-label errors can remain useful despite being class-incorrect.

Research on anti-forgetting training strategies for open-vocabulary object detection systems

Open-vocabulary object detection leverages vision-language pre-training to achieve cross-category generalization. With the progressive refinement of models such as GLIP, Grounding DINO, and YOLO-World, detectors have exhibited substantially enhanced zero-shot recognition capabilities on open vocabularies, with YOLO-World further advancing this capability into an efficient single-stage detection paradigm. However, during scenario-specific fine-tuning, training that relies solely on ground-truth annotations of base classes induces rapid degradation in the model’s discriminative ability for novel classes, often reverting to closed-set detector behavior. To address this issue, the present paper proposes a forgetting-resistant fine-tuning framework that decouples the training objectives into strong supervised adaptation for base classes and weak supervised preservation for novel classes. Specifically, novel pseudo-boxes are generated once by a frozen zero-shot teacher before any task-specific fine-tuning and are refined through threshold filtering, score weighting, and soft-ignore mechanisms to reduce the impact of noisy pseudo-supervision. Subsequently, teacher-guided logit distillation is incorporated, constraining the student model exclusively at the output distribution level to preserve its response relationships to the novel vocabulary. Experimental results demonstrate that conventional fine-tuning on YOLO-World-v2-S reduces Novel AP50 from 57.6 to 1.6. In contrast, through stepwise integration of the freezing strategy, pseudo-labels, and logit distillation, the proposed method elevates Novel AP50 to 51.6 and achieves an All AP50 of 53.7. These findings show that the proposed approach substantially mitigates catastrophic forgetting while preserving the original single-stage inference pipeline.

SAM3-LoRA: Parameter-Efficient Adaptation of a Concept-Promptable Foundation Model for Multi-Class Structural Defect Segmentation

Promptable segmentation foundation models such as SAM3 accept an open-vocabulary text concept and return every instance matching it, but adapting them to a specialized domain by full fine-tuning is computationally prohibitive for the organizations that would benefit most. This study applies Low-Rank Adaptation (LoRA) to SAM3 for multi-class structural defect segmentation and examines both how such a model can be supervised from conventional annotation and whether the resulting efficiency gain transfers across datasets. Two contributions are methodological. First, we describe a supervision procedure that trains a concept-promptable model directly from COCO-style class-labeled instance segmentation by using the category name itself as the prompt, requiring no prompt templates, no synonym expansion, and no learned class embeddings. Second, we identify and mitigate a failure mode specific to this setting: because a conventional annotation file yields positive prompts exclusively, the model's presence prediction decouples from the text condition and degenerates into responding to any prompt, a collapse that is invisible to every metric computed on positive prompts alone. Exhaustive hard-negative prompting, in which every dataset category absent from an image is issued as a zero-detection query, addresses this at no annotation cost. Two adapter placements were compared under an identical protocol, updating 0.121% and 1.341% of model parameters. On a purpose-built tunnel lining dataset, pixel intersection-over-union improved from 0.017 to 0.338 and instance-level recall from 0.375 to 0.672; on the independent public Structural Defects Dataset, from 0.017 to 0.855 and from 0.574 to 1.000. Improvements were directionally consistent across ten metrics on both datasets, and the largest per-category gains occurred precisely where zero-shot competence was absent.

VCF-CLIP: Visual Context-Driven Fine-Grained Prompt Learning for Zero-Shot Anomaly Detection.

This work proposes VCF-CLIP, a visual context-driven fine-grained prompt learning framework built upon CLIP, and proposes the prompt prototype learning (PPL) strategy, which learns a pair of unified prompt prototypes representing general normal and anomalous states in a loss-guided manner, thereby eliminating the need for manual prompt design.

Tiper: Tokenwise Prompt-Free Learning

Vision-language models (VLMs), such as contrastive language-image pre-training (CLIP), exhibit powerful zero-shot generalization capabilities. Parameter-efficient fine-tuning (PEFT) techniques, notably prompt learning, have been extensively explored to adapt these models to downstream tasks. However, their efficacy remains constrained when transferred to specialized domains like remote sensing. We argue that the bottleneck stems not merely from the limited parameters of prompts, but essentially from the disruption of the input’s original image–text features and the lack of deep cross-modal alignment. In particular, existing methods typically rely on global attention or coarse-grained feature mapping. This inadvertently corrupts the original input representations, thereby impairing the model’s inherent generalization. Furthermore, their isolated unimodal gradient updates fail to bridge the semantic gap inherent in complex remote sensing scenes. To address these challenges, we propose tokenwise prompt-free learning (Tiper), shifting the optimization paradigm from introducing external prompts to precisely recalibrating the critical tokens that govern classification outputs. In particular, Tiper employs a hierarchical learner to supersede global prompts. Crucially, this learner intervenes exclusively on the specific core tokens (i.e., the CLS token in the visual branch and the EOT token in the textual branch), leaving other original input representations unperturbed. This fine-grained strategy effectively balances domain adaptation with the preservation of inherent generalization. Finally, we design the learner as a cross-modal coupled bridge with shared weights, enabling it to synchronously receive gradient feedback from both modalities and fostering profound multimodal collaboration. Extensive experiments validate our method on eight public remote sensing datasets covering diverse scenes and resolutions. In the base-to-new generalization task, Tiper outperforms the strong baseline MaPLe with a significant 3.7% improvement in the harmonic mean (HM). Notably, without relying on any external large-scale domain models, Tiper surpasses the latest domain-specific prompt learning methods (e.g., domain-controlled prompt learning (DCPL), domain prompt learning with quaternion networks (DPLQ)), demonstrating its superior adaptability for remote sensing image scene classification.

Hierarchical prompt composition for memory-efficient open-world continual learning in vision-language foundation models