Skip to content

A cross-modal fusion detection framework for robust object perception under challenging illumination

Aug 2026 · Measurement science and technology · Vol 37, pp. 325406 · 0 citations · 44 references
Physics

TL;DR

A cross-modal fusion (CMFusion) detection framework is proposed, which focuses on efficient multi-level RGB-IR feature interaction within YOLO-based detectors, enabling effective cross-modal complementarity with lower computational overhead.

Abstract

RGB-infrared (RGB-IR) fusion improves object detection by enabling robust object localization under challenging illumination and environmental conditions. However, the additional IR modality often increases computational cost, limiting its deployment in real-time measurement and perception systems. This work revisits cross-modal fusion strategies and shows that strong performance can be achieved by progressively accumulating efficient cross-modal interactions across multiple feature scales, without duplicating auxiliary modality feature-extraction branches. To exploit this observation, a cross-modal fusion (CMFusion) detection framework is proposed, which focuses on efficient multi-level RGB-IR feature interaction within YOLO-based detectors. This design reduces redundant feature extraction within individual modalities while enhancing multi-level multimodal interaction, enabling effective cross-modal complementarity with lower computational overhead. Extensive experiments on three public RGB-IR datasets across two detection tasks (OBB and HBB) demonstrate that CMFusion achieves a 1.7%–6.1% mAP improvement over baseline unimodal detectors while maintaining comparable low computational complexity, and consistently surpasses state-of-the-art methods.

View source

Similar papers

Open access 2026

Frequency-Guided Cross-Modal Distillation With Illumination-Invariant Prototype Alignment for RGB-Infrared Object Detection

Multi-modal object detection using red-green-blue (RGB) and infrared (IR) imagery is essential for safety-critical applications under diverse illumination conditions, such as autonomous driving and urban surveillance. Recent RGB-IR detectors have improved detection performance through feature-level fusion strategies, i...

Minju Baek, Eunseong Lee, Jaehong Yoon et al. · 0 citations
Open access Nov 2026

Lightweight cross-modal RGB-IR fusion network for real-time aerial surveillance

Real-time onboard object detection is essential for autonomous UAV applications requiring rapid and reliable decision-making in dynamic environments. Most existing onboard detectors rely on RGB imagery due to its computational efficiency; however, RGB-based models suffer significant performance degradation under low-il...

N. Habash, Ahmad Abu Alqumsan, Tao Zhou · 0 citations
Conference Sep 2026

A low-light RGB-thermal infrared target detection algorithm based on adaptive cross-modal fusion and multiscale feature enhancement

Small-object detection under low illumination remains a persistent challenge in aerial surveillance, nighttime patrol, and safety-critical vision tasks, where single-modality sensors—RGB or infrared alone—provide degraded or incomplete target information. This paper proposes an RGB-T object detection network that combi...

Xin Song, Jian-Jun Luo, Ping Wang · 0 citations
Conference Nov 2026

Object detection algorithm in complex environments based on multimodal sensor fusion and deep learning

Accurate object detection in complex environments involving adverse weather, occlusion, and insufficient illumination poses a persistent challenge for autonomous driving and intelligent surveillance systems. Single-modality approaches based on cameras or LiDAR alone often fail under these conditions due to inherent sen...

Yanjun Pu · 0 citations
Open access 2026

MAMENet: Modal Alignment and Multiscale Feature Enhancement Network for RGB–IR Small Object Detection

Object detection using visible–infrared images has become increasingly important for all-day detection scenarios. However, due to significant imaging discrepancies between the visible and infrared modalities, achieving accurate modal alignment and effective feature fusion remains a major challenge. Existing methods oft...

Kai-Yue Men, Cheng-You Wang, Xiao Zhou et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.