SCAF-Net: Sparse Cross-Modal Adaptive Fusion Network for RGB-IR UAV Object Detection

Abstract

Visible-infrared (RGB-IR) object detection equips uncrewed aerial vehicles (UAVs) with robust, around-the-clock perception capabilities. However, practical deployments are frequently constrained by weak spatial misalignment between heterogeneous sensors. This issue degrades the object detection accuracy of UAVs, for which a modest offset may cause corresponding features to overlap background regions or neighboring instances. In addition, the substantial appearance difference between RGB and infrared imagery makes direct cross-modal matching unreliable. Inspired by feature decoupling paradigms and cross-modal attention mechanisms, we propose a geometry-guided RGB-IR framework termed the sparse cross-modal adaptive fusion network (SCAF-Net) to improve feature correspondence and interaction under such conditions. Specifically, the modality-decoupled offset estimator estimates modality-shared geometric representations and bidirectional offsets. The geometry-aware sparse cross-attention module uses these offsets to perform saliency-guided sparse cross-modal attention with a coordinate-distance bias, while the context-preserving bidirectional warping mechanism reintegrates the enhanced sparse tokens into dense feature maps within the same reference coordinate domain. The key novelty is the unified offset-guided coordinate flow that consistently guides correspondence estimation, token interaction, and feature reconstruction. Extensive experiments on the DroneVehicle and RGBTDronePerson datasets demonstrate that SCAF-Net achieves competitive performance, exhibiting enhanced robustness in complex aerial-view scenarios.