The Clustered Unit-level Similarity Transformer (CUST) is proposed, a novel architecture that efficiently integrates global and local information and achieves a lower memory footprint and faster inference speed compared to recent global context or lightweight models under realistic constraints.
Abstract
Recently, Vision Transformer (ViT)-based models have exhibited remarkable performance in image super-resolution. However, the quadratic computational complexity of ViTs with respect to spatial resolution severely constrains their efficiency, leading to high latency and massive memory consumption. To alleviate this, various window-based attention mechanisms have been proposed; yet, they inherently compromise the long-range dependency modeling that is the primary advantage of ViTs. To overcome these limitations, we propose the Clustered Unit-level Similarity Transformer (CUST), a novel architecture that efficiently integrates global and local information. Specifically, CUST enables each patch to aggregate and attend to similar patches within a broadened regional scope outside its local window, thereby capturing extensive contextual understanding. Furthermore, it employs overlapping attention windows to capture local dependencies, while explicitly extracting high-frequency details by computing the residual difference between the original features and their downsampled-upsampled counterparts. Comprehensive experiments demonstrate that our proposed model achieves a practical balance between computational efficiency and restoration performance. It achieves a lower memory footprint and faster inference speed compared to recent global context or lightweight models under realistic constraints. Code is available at [https://github.com/jwgdmkj/CUST].
A lightweight dual-dimension modulation aggregation network, which combines channel-wise and spatial feature interactions to achieve more accurate reconstruction, and shows that DMANet achieves competitive reconstruction performance with lower model complexity and runtime overhead.
Fanping Liu, Bendu Bai· Signal, Image and Video Proc...· 0 citations
Semantic segmentation demands a careful balance between accuracy, efficiency, and scalability, which remains difficult to achieve for high-resolution imagery. Convolutional networks effectively model local patterns but struggle with long-range dependencies, whereas Vision Transformers capture global context at a high computational cost. While recent work largely focuses on encoder design, the bottleneck stage, central to contextual aggregation and information flow, has been relatively overlooked. We propose SiConMo, a lightweight yet effective framework, implemented in two variants: an RGB-only model (SiConMo) and a GME-enhanced variant (SiConMo$_\dagger$). We show that simplicity arises from a key design principle: at very low computational budgets, the bottleneck is the most efficient stage to integrate local and global context. SiConMo integrates three complementary components: a Token Pyramid Extraction Module for hierarchical multi-scale representation, a Transformer-Branched Depthwise Convolution block for bottleneck-aware context modeling, and a Feature Merging Module that preserves spatial structure while enhancing semantic consistency. Extensive experiments on ADE20K, PASCAL Context, Cityscapes, and COCO-Stuff demonstrate that SiConMo achieves a state-of-the-art accuracy-efficiency trade-off among lightweight semantic segmentation models, highlighting simplicity as a powerful design principle.
Mian Muhammad Naeem Abid, Nancy Mehta, Zongwei Wu et al.· International Conference on...· 0 citations
An efficient aligned kernel network (AKNet) is proposed, which innovatively employs super-large convolution kernels to capture global receptive fields with minimal computational overhead, effectively mimicking the long-range dependency modeling of transformers.
Wan Li, Xiao-Lin Zhang· The Visual Computer· 0 citations
The results indicate that the collaboration between global semantics and local details within a unified weighting domain can effectively improve the separability and deploy ability of high-resolution segmentation.
Yuyang Wang, Jia-Mei Hu, Xinwei Wang et al.· International Conference on...· 0 citations
The Visual Transformer (ViT) has demonstrated powerful capabilities in modeling patch-wise attention for image classification. However, existing approaches typically treat all image regions uniformly, neglecting their inherent differences in importance. To address this limitation, we propose the Adaptive Attention Region Transformer (AART), which dynamically discriminates between regions based on their saliency. Our method begins by identifying key regions through density analysis of feature points, where the centroid of the densest cluster defines attention regions, with remaining areas designated as non-attention regions. We then implement differentiated feature extraction: small convolutional kernels capture fine-grained details from attention regions, while large kernels extract coarse-grained features from non-attention regions. This multi-scale feature extraction strategy enables more efficient representation learning. The resulting features are integrated and processed through Transformer blocks to learn comprehensive self-attentive representations. Extensive evaluations on CIFAR-10 and CIFAR-100 demonstrate that AART achieves significant improvements over existing methods without requiring pre-training, validating its effectiveness in adaptive region processing.
