Jul 2026· International Conference on Ubiquitous and Future Networks· pp. 1056-1061· 0 citations· 11 references
Abstract
Transfer learning with ImageNet-pretrained backbones is ubiquitous in computer vision, yet the question of which intermediate layer best serves a downstream task is typically resolved by convention rather than controlled measurement. Despite the prevalence of this practice, no systematic comparison of layer-wise frozen representations has been conducted across the modern architectures that dominate current pipelines. We present the Layer Transferability Profile, a reproducible evaluation protocol and an accompanying metric, the Layer-Task Transferability Score (LTTS), that together quantify how transferable information is distributed across depth. Applying the profile to three architectures, ResNet-50, EfficientNet-B0, and ViT-B/16, on three recognition benchmarks (CIFAR-10, CIFAR-100, and Food101), we find that transfer performance improves monotonically with depth and that the final or near-final representation is consistently optimal (LTTS 0-1.2%). This pattern contrasts with recent findings in speech and language foundation models, where intermediate layers can yield substantial gains, suggesting that the concentration of transferability in late layers is characteristic of supervised visual pretraining for recognition tasks rather than a universal property of deep representations. Notably, ViT-B/16 achieves the strongest transfer accuracy (78.1% on CIFAR-100, 94.6% on CIFAR-10, and 68.6% on Food-101) while using 768-dimensional features, outperforming ResNet-50’s 2048-dimensional and EfficientNet-B0’s 1280-dimensional representations. This indicates that representational quality per dimension varies substantially across architectures and can be more important than raw feature size when selecting a backbone for transfer.
Learning rate (LR) initialization and decay remain important factors in the optimization of deep vision networks. Although these models exhibit a clear hierarchical structure, training typically starts from a single global learning rate, with little explicit consideration of stage depth. This paper investigates a simple depth-aware alternative. We propose Asynchronous Learning Rate (ALR), which assigns depth-dependent initial learning rates to network modules according to their topological depth, and Smoothed Synchronous Decay (SSD), which coordinates the subsequent decay of heterogeneous parameter groups. A linear depth rule is adopted as a low-complexity, monotonic parameterization that is straightforward to implement across hierarchical architectures. The proposed strategy is evaluated on image classification and semantic segmentation benchmarks, including CIFAR-100, Mini-ImageNet, ImageNet-1K, Pascal VOC 2012, and LiTS. Additional ResNet-50 evaluations under benchmark-specific training protocols show that depth-scaled ALR improves Top-1 accuracy from 77.618% to 78.736% on native CIFAR-100 across five seeds and from 76.33% to 78.58% on ImageNet-1K. The results indicate that ALR generally outperforms uniform learning rate initialization, whereas SSD is most effective when combined with depth-aware initialization, particularly when layer-wise learning rates would otherwise follow misaligned decay trajectories. These findings suggest that a simple hierarchy-aware learning rate design can serve as an effective optimization refinement for hierarchical vision models.
Qiang He, Qiu Zong, Yi-Qi Wang et al.· IEEE Access· 0 citations
Vision foundation models are increasingly reused as frozen backbones for downstream visual recognition, making parameter-efficient adaptation a central problem. Prompt-based adaptation, including Visual Prompt Tuning (VPT), provides a lightweight way to specialize these models, but its layer-wise behavior remains poorly understood: performance is sensitive to prompt depth, placement, and task distribution, and gains on standard in-domain benchmarks do not always translate into robust generalization. We argue that this limitation is not solely an optimization issue, but a layer-wise information allocation issue: existing prompt-based methods lack principled control over what prompt-conditioned representations should preserve, suppress, and propagate across depth. Inspired by the Information Bottleneck principle, we introduce Prompted Information Bottlenecks (PIB), a framework that regularizes layer-wise compression-sufficiency trade-offs and promotes a more coherent cross-layer information path. The key idea is that effective adaptation should be minimal yet sufficient, retaining task-relevant local evidence in earlier layers while progressively discarding nuisance factors and redundant details in deeper layers. Extensive experiments show that PIB achieves strong performance across 34 datasets, reaching 92.1% on FGVC, 93.01% on HTA, and 77.33% on VTAB-1k, while tuning only 0.35% parameters on average across the main settings. Beyond benchmark accuracy, PIB helps explain the non-monotonic behavior of prompt capacity scaling, reduces shortcut reliance, and improves robustness under distribution shift and fine-grained recognition settings. These results position PIB as both a practical method and an information-allocation perspective for adapting frozen vision foundation models. Our code is available at https://github.com/itsnotacie/MM-26-PIB
Yuqi Li, Xi Xiao, Yunbei Zhang et al.· arXiv.org· 4 citations
Across cross-generator, post-processing, and in-the-wild benchmarks, PE-SPC surpasses the previous DINOv3 baseline and achieves new state-of-the-art results.
Wei-Han Cai, Hao Tan, Zichang Tan et al.· 0 citations
This work introduces \method, a Multi-scale Adaptive Vision Encoder, a Multi-scale Adaptive Vision Encoder that uses position-dependent gates to fuse shallow, intermediate, and deep features from a vision Transformer, preserving global semantics while enhancing edges, text, and local structure.
LLaVA-style Vision-Language Models (VLMs) pass visual tokens from a fixed late layer of the vision backbone, typically the penultimate one, to the language model. We first show that this hidden convention is fragile: across 2 VLMs and 7 image and video benchmarks, the default layer is sub-optimal in 13 of 14 model-task pairs, and the best layer shifts with both task and visual backbone. Finding that layer by exhaustive layer-wise inference is prohibitively expensive, and no better fixed default exists. We therefore ask whether layer usefulness can instead be predicted from representation geometry. We study matrix-based entropy, introduced for unimodal layer analysis, which we compute over sample-level visual embeddings as Visual Dataset Entropy (VDE); and Gromov-Wasserstein (GW) distance, introduced for encoder-level VLM model selection, which we repurpose as a layer-wise visual--language alignment signal. Transferring these to LLaVA-based models is not obvious a priori: the vision tower is frozen while the multimodal projector is trained, so we profile both sides of the projector. We find that VDE transfers, and GW does not. Computed from 100 unlabeled task samples without downstream inference, pre-projector VDE tracks layer-wise accuracy and its top-ranked layers cover the oracle best layer on every task for the SigLIP-based LLaVA-Video, while giving region-level guidance for the CLIP-based Video-LLaVA. Post-projector profiles show that the projector reshapes visual geometry but does not erase the performance-relevant trend, leaving $\mathrm{VDE}_{\mathrm{pre}}$ the stronger signal. GW instead flattens after projection and is best read as an alignment diagnostic rather than a selector. VDE thus offers an interpretable, training-free policy that narrows the visual-layer search to a handful of candidates for limited downstream verification.
Ruchen Liu, Yi Yang, Yiming Xu et al.· 0 citations
The Robust and Fine-grained training framework for CLIP-based vision-language models (RoFLIP) is proposed, enhancing both the robustness and granularity of vision-language alignment and underscore RoFLIP’s compositional reasoning and generalization abilities.
Yiwei Sun, Chuanbin Liu, Shancheng Fang et al.· International Journal of Com...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
