Cross-Modal Mamba-Transformer Fusion for AAV-Based Visible-Infrared Tiny Object Detection

Abstract

Autonomous aerial vehicles (AAV)-based visible-infrared (RGB-IR) fusion is crucial for tiny object detection in search and rescue and surveillance. However, detecting extremely tiny objects that occupy few pixels and are easily confounded by background interference remains challenging. While existing fusion methods have made progress, they often fail to adequately account for the unique characteristics of tiny objects. To address this, we propose a cross-modal Mamba-Transformer fusion (CMMTF) framework that mitigates two core limitations: insufficient exploitation of complementary frequency-domain information and inadequate adaptive feature selection. First, we introduce a frequency-aware cross-modal transformer with differential interaction (FACT-DI) strategy that decomposes RGB and IR features via wavelet transform, computes and enhances intermodal differential features, and employs them as queries in cross-modal attention. Second, we design a convolution-channel guided feature enhancement Mamba (CCFE-Mamba) mechanism that leverages Mamba's global modeling for holistic feature selection and refines features through local spatial-channel recalibration, adaptively accentuating the most informative features for tiny objects. Experiments on RGBTDronePerson and RGBT-Tiny show that CMMTF achieves state-of-the-art performance, outperforming existing methods by 2.4 percentage points in $AP_{50}^{tiny}$ on RGBTDronePerson and by 2.5 percentage points in $AP_{50}^{all}$ on RGBT-Tiny. Our framework provides a robust solution for advancing tiny-object detection in complex AAV scenarios.