Object detection algorithm in complex environments based on multimodal sensor fusion and deep learning
Abstract
Accurate object detection in complex environments involving adverse weather, occlusion, and insufficient illumination poses a persistent challenge for autonomous driving and intelligent surveillance systems. Single-modality approaches based on cameras or LiDAR alone often fail under these conditions due to inherent sensor limitations. This paper proposes CMAFNet (Cross-Modal Attention Fusion Network), a novel multimodal sensor fusion framework that jointly leverages camera images and LiDAR point clouds for robust 3D object detection. CMAFNet employs a dual-branch architecture with ResNet-50 for 2D image feature extraction and PointNet++ for 3D point cloud feature extraction, followed by a Cross-Modal Attention Fusion (CMAF) module that adaptively aggregates complementary information from both modalities via cross-attention mechanisms. The fused features are subsequently transformed into a Bird's Eye View (BEV) representation and passed to a multiscale detection head. Comprehensive evaluations on the KITTI and nuScenes benchmarks verify that CMAFNet achieves 88.14% AP for car detection on KITTI (moderate) and 69.8mAP on nuScenes, outperforming existing methods including BEVFusion by 1.22% and 1.3mAP respectively. Ablation experiments validate the contribution of each component, with the CMAF module providing a 3.3% improvement over naive early fusion.