Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 1886-1897· 0 citations· 12 references
Abstract
Prompt-based learning has emerged as a promising paradigm for Class Incremental Learning (CIL), enabling pre-trained models to adapt efficiently to open-world scenarios. Existing methods often employ correlation-based strategies, where an image's feature serves as a query to retrieve the most relevant key prompts, with corresponding value prompts for training. However, these approaches face a fundamental challenge: fitting the entire feature space of all tasks with only a few trainable prompts severely limits the pre-trained model's retrieval capability. In this paper, we propose Predictive Prompting (PrePrompt), a novel CIL framework that circumvents correlation-based limitations by leveraging the inherent classification ability of pre-trained models to predict task-specific prompts. Specifically, PrePrompt decomposes CIL into a two-stage prediction process: task-specific prompt prediction followed by a label prediction. While theoretically sound, this framework risks bias toward recent classes due to missing historical information for calibrating older classifiers. To mitigate this, PrePrompt incorporates a feature extrapolation technique, dynamically balancing stability and plasticity across classifiers. Extensive experiments on several benchmarks demonstrate PrePrompt's superiority over state-of-the-art prompt-based CIL methods. Code is available at https://github.com/libo-huang/preprompt.
P predictive Prompting (PrePrompt) is proposed, a novel CIL framework that circumvents correlation-based limitations by leveraging the inherent classification ability of pre-trained models to predict task-specific prompts and decomposes CIL into a two-stage prediction process: task-specific prompt prediction followed by a label prediction.
Libo Huang, Xiang-Qi Li, Jia-Rui Zhao et al.· Proceedings of the 32nd ACM...· 4 citations
Continual learning faces the persistent challenge of catastrophic forgetting, where sequential task updates degrade previously acquired knowledge. While prompt-based methods integrated with pre-trained models offer a compelling solution by freezing the backbone, they often rely on static, task-level prompting strategies that overlook fine-grained intra-task diversity. In this paper, we propose Gated Adaptive Prompting (GAP-Prompt), a novel method that introduces instance-level adaptability to the prompting process. GAP-Prompt consists of three synergistic modules: (1) instance-conditioned gating, which dynamically determines optimal prompt injection layers for each individual image; (2) dynamic knowledge fusion, which performs instance-aware aggregation of current and historical prompts, enabling knowledge integration across tasks; and (3) shared prompt distillation, which anchors foundational knowledge in early shared layers to mitigate forgetting. Extensive evaluations on CIFAR-100, ImageNet-R, and CUB-200 benchmarks demonstrate that GAP-Prompt consistently achieves state-of-the-art performance. Notably, on the fine-grained CUB-200 dataset, GAP-Prompt reaches 87.29% accuracy, approaching the joint training upper bound (88.00%) and outperforming existing methods by a significant margin.
Trung-Anh Dang, D. Bùi, Ngoc-Son Vu et al.· 0 citations
Vision-language models (VLMs), such as contrastive language-image pre-training (CLIP), exhibit powerful zero-shot generalization capabilities. Parameter-efficient fine-tuning (PEFT) techniques, notably prompt learning, have been extensively explored to adapt these models to downstream tasks. However, their efficacy remains constrained when transferred to specialized domains like remote sensing. We argue that the bottleneck stems not merely from the limited parameters of prompts, but essentially from the disruption of the input’s original image–text features and the lack of deep cross-modal alignment. In particular, existing methods typically rely on global attention or coarse-grained feature mapping. This inadvertently corrupts the original input representations, thereby impairing the model’s inherent generalization. Furthermore, their isolated unimodal gradient updates fail to bridge the semantic gap inherent in complex remote sensing scenes. To address these challenges, we propose tokenwise prompt-free learning (Tiper), shifting the optimization paradigm from introducing external prompts to precisely recalibrating the critical tokens that govern classification outputs. In particular, Tiper employs a hierarchical learner to supersede global prompts. Crucially, this learner intervenes exclusively on the specific core tokens (i.e., the CLS token in the visual branch and the EOT token in the textual branch), leaving other original input representations unperturbed. This fine-grained strategy effectively balances domain adaptation with the preservation of inherent generalization. Finally, we design the learner as a cross-modal coupled bridge with shared weights, enabling it to synchronously receive gradient feedback from both modalities and fostering profound multimodal collaboration. Extensive experiments validate our method on eight public remote sensing datasets covering diverse scenes and resolutions. In the base-to-new generalization task, Tiper outperforms the strong baseline MaPLe with a significant 3.7% improvement in the harmonic mean (HM). Notably, without relying on any external large-scale domain models, Tiper surpasses the latest domain-specific prompt learning methods (e.g., domain-controlled prompt learning (DCPL), domain prompt learning with quaternion networks (DPLQ)), demonstrating its superior adaptability for remote sensing image scene classification.
Tengfei Gong, Jun-Lin Wu, Yaxioong Chen et al.· IEEE Transactions on Geoscie...· 0 citations
This work introduces C-GAP Caption-Guided Augmentation and Prompting), a detector-agnostic, annotation-free framework that operates in two phases, and establishes a composite caption baseline combining per-image scene descriptions with class-quantity context, which is shown to outperforms scene-description only or class-quantity-only prompts across multiple open-vocabulary architectures and benchmarks.
Promptable segmentation foundation models such as SAM3 accept an open-vocabulary text concept and return every instance matching it, but adapting them to a specialized domain by full fine-tuning is computationally prohibitive for the organizations that would benefit most. This study applies Low-Rank Adaptation (LoRA) to SAM3 for multi-class structural defect segmentation and examines both how such a model can be supervised from conventional annotation and whether the resulting efficiency gain transfers across datasets. Two contributions are methodological. First, we describe a supervision procedure that trains a concept-promptable model directly from COCO-style class-labeled instance segmentation by using the category name itself as the prompt, requiring no prompt templates, no synonym expansion, and no learned class embeddings. Second, we identify and mitigate a failure mode specific to this setting: because a conventional annotation file yields positive prompts exclusively, the model's presence prediction decouples from the text condition and degenerates into responding to any prompt, a collapse that is invisible to every metric computed on positive prompts alone. Exhaustive hard-negative prompting, in which every dataset category absent from an image is issued as a zero-detection query, addresses this at no annotation cost. Two adapter placements were compared under an identical protocol, updating 0.121% and 1.341% of model parameters. On a purpose-built tunnel lining dataset, pixel intersection-over-union improved from 0.017 to 0.338 and instance-level recall from 0.375 to 0.672; on the independent public Structural Defects Dataset, from 0.017 to 0.855 and from 0.574 to 1.000. Improvements were directionally consistent across ten metrics on both datasets, and the largest per-category gains occurred precisely where zero-shot competence was absent.
P. Malaisree, S. Youwai, S. Janrungautai et al.· 0 citations
Miles decouples the learnable modules with the pre-trained model, exploiting prior information from intermediate features of the backbone network to enable more flexible parameter expansion, and orchestrating an efficient expansion of the parameter space through guided optimization.
Kai Jiang, Zisong Lin, Hongyuan Zhang et al.· IEEE Transactions on Image P...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
