Preprint

PPOM: Marginalizing Patch-Grid Phase for CLIP-Based Generalizable Vision-Language Prompt Tuning

Liang Wang Haoyang Li Chao Wang Guodong Long Jing Jiang Yan Peng

Aug 2026 · 0 citations · 17 references

Computer Science

TL;DR

Patch-Phase Orbit Marginalization (PPOM), a training-free inference operator that treats phase shift as a nuisance variable, provides a deterministic interface between prompt adaptation and patch-grid sensitivity.

Abstract

Prompt tuning adapts CLIP-based vision-language models with few trainable parameters, yet its predictions remain sensitive to the spatial sampling imposed by a frozen vision transformer. In particular, non-overlapping patch tokenization makes predictions depend on the alignment (phase) between image and the patch lattice. To reduce prediction sensitivity to patch-grid alignment, we introduce Patch-Phase Orbit Marginalization (PPOM), a training-free inference operator that treats phase shift as a nuisance variable. Given a patch stride, PPOM evaluates the identity view and reflection-padded translations, pairs opposite shifts into horizontal, vertical, and diagonal antithetic families, and assigns equal mass to these families and the identity prediction to avoid view-count bias during phase integration. In summary, PPOM provides a deterministic interface between prompt adaptation and patch-grid sensitivity. Across multiple prompt-learning hosts, PPOM improves host performance without re-training.

arXiv Semantic Scholar

View source

Similar papers

#small language model Preprint Sep 2026

TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models

TecoPrompt, a closed-loop robust prompt-learning framework that revisits optimal transport pseudo-labeling from a temporal perspective, employs an entropic OT plan in the CLIP semantic space to obtain globally consistent label candidates.

Z. Shao, Hao-Wen Hua, Jia-Xin Zhang et al. · 0 citations

Preprint Oct 2026

TraceICL: Thread-Resolved Demonstration Curation with Consistency Calibration for Robust Multimodal In-Context Learning

Interleaved multimodal in-context learning (ICL) lets frozen large vision-language models (LVLMs) acquire new tasks from a handful of image-text-answer demonstrations, yet it remains fragile: demonstrations bind to the wrong images, redundant exemplars crowd out informative ones, and noisy answers are copied verbatim i...

Y. Yılmaz, Elena Petrova, M. Kaya et al. · 0 citations

Preprint Oct 2026

LAMP: Layerwise Adaptive Mixing of Predictors for Training-Free Few-Shot Vision-Language Adaptation

Few-shot adaptation often relies on an encoder's final global representation, leaving complementary information in intermediate layers underused. Combining these features is challenging because their dimensions can differ and their usefulness varies across tasks. We propose LAMP, Layerwise Adaptive Mixing of Predictors...

Zhi-Peng Ye, Feng Jiang, Qiu-Feng Wang et al. · 0 citations

#artificial intelligence Preprint Sep 2026

Re-calibrated Contrastive Loss for Transformation-Aware Prompt Conditioning in Vision-Language Models

This work addresses limitations in transfer learning for vision-language models through transformation-aware prompt conditioning and a re-calibrated contrastive loss, and treats same-class samples as positives rather than distinct instances, enabling the model to learn domain-specific features more effectively.

Seungmin Oh, Seung-Hun Kang, Jongbin Ryu · 0 citations

#artificial intelligence Preprint Sep 2026

Domain Recentering and Confidence-Weighted Prior Calibration for Vision-Language Models

Vision-language models such as CLIP achieve strong zero-shot classification, yet under distribution shift, visual embeddings drift from fixed text embeddings. Training-free calibration avoids the per-sample optimization of prompt learning, but prior feature calibration gives each image the full bias of one hard cluster...

Youngeun Seol, Jiyon Shin, Hee Suk Yoon et al. · 0 citations

#small language model Preprint Sep 2026

ProCAP: Probabilistic Cross-Attentive Prompt Learning for Vision-Language Models

ProCAP is proposed, a probabilistic cross-attentive prompt learning framework that improves cross-modal interaction and training stability without updating any CLIP weights: it learns both visual and textual prompt tokens and links them through stacked bidirectional multi-head cross-attention so the two branches refine...

Hiwa Azeez Abbas, Fatemeh Daneshfar, M. Abdar · 0 citations