Aug 2026· Journal of King Saud University: Computer and Information Sciences· Vol 38· 0 citations· 48 references
TL;DR
This work proposes AMCA, an unbiased SGG framework integrating adaptive multi-prototype learning with cross-modal alignment, which achieves consistently competitive performance across multiple SGG tasks, with particularly strong improvements on the unbiased mR@K metric.
Abstract
Scene graph generation (SGG) aims to detect objects and recognize their semantic relationships from images, serving as an important bridge between visual perception and high-level semantic understanding. Existing debiasing methods primarily address inter-class long-tailed bias, while paying insufficient attention to intra-class semantic heterogeneity of predicates. High-frequency predicates typically encompass multiple semantic sub-patterns, causing instances of the same predicate to scatter widely in the visual feature space and resulting in a fundamental misalignment between feature distributions and classification objectives. To address these issues, we propose AMCA, an unbiased SGG framework integrating adaptive multi-prototype learning with cross-modal alignment. Specifically, the Multi-Prototype Construction (MPC) module employs a distribution-aware prototype allocation strategy to quantify the semantic distributional complexity of each predicate and dynamically assign prototype counts, explicitly modeling intra-class semantic heterogeneity. Building on this, the Hybrid Relation Refinement Module (HRRM) embeds semantic prototypes into the visual relation feature refinement process via a prototype-augmented attention mechanism, and further employs Mamba to model global contextual dependencies among different relation representations, achieving deep fusion of semantic priors and visual features. Furthermore, the Visual-Text Alignment (VTA) module aligns the fused visual relation representations with CLIP text embeddings, significantly enhancing the model’s ability to distinguish fine-grained semantics. Extensive experiments on Visual Genome and GQA demonstrate that AMCA achieves consistently competitive performance across multiple SGG tasks, with particularly strong improvements on the unbiased mR@K metric. Our code is publicly available at https://github.com/SJVision/AMCA.
Scene graph generation (SGG) addresses the task of detecting objects in an image and predicting the relationships among them. Although prototype-based methods have recently achieved clear progress on long-tailed SGG, fine-grained low-frequency predicates remain difficult to recognize because their relation features often exhibit larger intra-class variation and more dispersed distributions, making them easily confused with semantically similar high-frequency coarse-grained predicates under a unified prototype-matching rule. To alleviate this issue, we propose a frequency-aware elastic prototype boundary learning framework, termed SGE-Net. Under fixed relation prototypes, the framework learns relation-category-specific boundary scales through explicit frequency compensation and frequency-adaptive virtual sampling, so that relation prediction can exploit not only prototype-center matching but also category-dependent decision-boundary information. During inference, we further introduce elastic boundary-aware distance calibration, enabling the boundary information learned during training to better distinguish relation categories that are easily confused under prototype matching. In addition, we combine visual and semantic features with dynamic gating to provide more reliable relation features for the above boundary learning. Experiments and analyses on Visual Genome and Open Images V6 demonstrate that the proposed method achieves consistent gains in both long-tailed relation prediction and overall evaluation metrics.
Binghao Wang, Xueying Sun, Hanzhu Dai et al.· Journal of King Saud Univers...· 0 citations
Fine-grained cross-modal understanding in drone views is essential for aerial vision-language navigation. However, the inherent wide field of view and overhead perspective of drone scenarios impose dual challenges on vision-language understanding. At the macro level, overwhelming background clutter in visual representations leads to Cross-Modal Focus Misalignment, where the model prioritizes global environmental similarities over specific object details. At the micro level, Visual Isomorphism creates ambiguity, where candidates share similar geometric structures yet differ only in subtle attributes. To address these challenges, we propose the Granularity-Aware Region Alignment and Semantic Prototype (GRASP) learning framework, enhancing discriminative capability through two synergistic strategies. Specifically, we introduce Region-Focused Alignment (RFA) to promote object-centric cross-modal alignment while suppressing background interference. Concurrently, to tackle visual isomorphism, we propose Semantic Perturbation Enhanced Matching (SPEM), which leverages a foreground-purified Semantic Prototype Codebook (SPC) to construct semantically perturbed negatives for fine-grained semantic discrimination. Extensive experiments on the GeoText-1652 benchmark and the unseen ERA dataset demonstrate that GRASP achieves competitive performance in drone-view fine-grained image-text retrieval, validating its effectiveness for cross-modal understanding in aerial scenarios. Our code implementation is available at https://github.com/UCAS-JC/GRASP.
Jiahui Cui, Yan Zhao, Kangye Wei et al.· 0 citations
The Cross-Modal Dual-Path Generator (CM-DPG), a model for robust open-vocabulary SGG that enhances object-level semantic representations through joint visual-textual encoding and improves relational reasoning using complementary visual and geometric cues, is proposed.
Minghao Zou, Qingtian Zeng, Shangkun Liu et al.· 0 citations
3D object detection stands as a pivotal task in scene understanding. However, two primary bottlenecks constrain current methodologies: semantic ambiguity arising from spatial misalignment during cross-modal fusion, and the inadequate contextual representation of individual candidate points within complex scenes. To address these challenges, this paper presents a Graph-driven Contextual Synergy Network (GCS3D), which is designed to systematically enhance point representations across both semantic and geometric dimensions. Specifically, the proposed method incorporates a Semantic Representation Rectification (G-SRR) module for cross-modal representation enhancement. By performing region-level semantic aggregation based on 3D neighborhoods to mitigate projection bias, this module achieves robust cross-modal fusion through a Spatial-aware Gating Mechanism (SGM) that adaptively regulates visual feature injection. Regarding contextual correlation modeling, the framework introduces a Graph-guided Geometric Consistency Interaction (G-GCI) module. By constructing a local topology graph among anchors and executing position-aware feature interaction, this module facilitates the aggregation of complementary neighborhood information, thereby bolstering the feature consistency and discriminability of anchor representations. Furthermore, a Spatial-Scale Aware Assigner (SSA-Assigner) is utilized to dynamically allocate supervision signals based on prediction quality, fully exploiting the performance potential inherent in the enhanced anchor representations. Extensive experiments on the SUN RGB-D and ScanNet V2 datasets demonstrate that GCS3D achieves superior results with mAP@0.25 scores of 70.39 and 73.86 respectively, validating the effectiveness and robustness of the proposed strategy in complex indoor scenes.
Vision foundation models (VFMs) offer strong generalization capabilities for domain-adaptive object detection (DAOD). However, existing VFM-based methods overlook the spatial-scale discrepancy between teacher and student feature maps, resulting in semantic incompatibility that weakens both feature alignment and pseudo-label learning. Moreover, domain shift can cause source-trained VFM teachers to miss target-domain objects, limiting the quality of their pseudo-labels. To address these issues, we propose the Semantic Localization-Enhanced Teacher (SLE-T), a semantically compatible knowledge-distillation framework built around a lightweight SLE Adapter for DINOv2. SLE Adapter injects pretrained local-texture priors into DINOv2 to improve cross-domain recognition and reformulates its features into dense representations that are spatially and semantically compatible with the student detector. SLE-T transfers the resulting teacher knowledge through either pseudo-label learning or feature alignment. We instantiate SLE-T with DINOv2-B and DINOv2-L (the ViT-B and ViT-L variants) and compare them with the larger DINOv2-G teacher. Extensive experiments on three DAOD benchmarks demonstrate that our method achieves state-of-the-art performance, and ablation studies confirm the importance of teacher-student semantic compatibility. Notably, SLE-T with DINOv2-B produces competitive or superior pseudo-labels using approximately one-quarter of the training time of DINOv2-G and substantially less GPU memory, demonstrating efficient VFM knowledge transfer under limited computational resources.
Qifeng Zhang, Ting Xiang, Ze-Yu Bai et al.· 0 citations
Bi-SGL is proposed, a Bidirectional, Spatially Grounded, and Language-informed framework for point cloud semantic scene completion that improves semantic completion accuracy among evaluated point cloud SSC baselines while using substantially fewer parameters than cascaded dense-fusion models.
Houda Saffi, N. Otberdout, Amal El Fallah Seghrouchni· IEEE Access· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
