2026· IEEE Open Journal of Vehicular Technology· Vol 7, pp. 2406-2437· 0 citations· 155 references
TL;DR
This survey addresses the fundamental challenge of deploying high-performance models for multimodal fusion in resource-constrained automotive environments, and organise state-of-the-art deep learning approaches into five paradigms—CNN-based, transformer-based, dense BEV-based, sparse-based, and hybrid—revealing trade-offs in accuracy, latency, and efficiency.
Abstract
Interaction with the physical world differentiates physical AI from other forms of AI. Autonomous driving exemplifies this; vehicles must perceive and respond to dynamic environments with human-like or better perception-reaction times. This survey addresses the fundamental challenge of deploying high-performance models for multimodal fusion in resource-constrained automotive environments. We organise state-of-the-art deep learning approaches into five paradigms—CNN-based, transformer-based, dense BEV-based, sparse-based, and hybrid—revealing trade-offs in accuracy, latency, and efficiency, as well as strengths and limitations in robustness under adverse operational design domains. The hardware-aware perspective is a differentiating contribution, presenting strategies for deployment on automotive platforms, reducing inference latency by up to 50% and improving robustness in adverse conditions by up to 20%. By synthesising sensor fusion, deep learning, compute platforms, and hardware-awareness, this work equips researchers and practitioners with actionable insights and strategies for perception systems, bridging theoretical advances and production-grade autonomous driving requirements.
The paradigm shift to "vehicle-road synergy" infrastructure as a critical mechanism for providing the safety redundancy and edge-computing capabilities needed for fully reliable, next-generation autonomous driving systems is explored.
Xichen Huang· Applied and Computational En...· 0 citations
Abstract. Vision-based object detection is a key component of autonomous driving perception systems; however, models pretrained on large-scale generic datasets usually struggles when implemented in automotive environments due to domain shift. This research introduces a comprehensive evaluation and fusion of YOLO11 and RT-DETR for improving robustness in autonomous driving scenarios using KITTI dataset. Both models are pretrained on COCO dataset and evaluated under a zero-shot transfer setting to assess cross-domain generalization. The results show that RT-DETR-L and RT-DETR-XL experience significant performance degradation, dropping from 53.0 and 54.8 𝑚𝐴𝑃 on COCO to 34.3 and 34.5 on KITTI, respectively. In contrast, YOLO11-Nano and YOLO11-L demonstrate better generalization, achieving 44.0 and 51.3 𝑚𝐴𝑃 on KITTI compared to 40.9 and 55.0 on COCO. Controlled fine-tuning experiments (10 and 100 epochs) are conducted to analyze adaptation dynamics. RT-DETR-L improves to 61.7 and 79.1 𝑚𝐴𝑃, while YOLO11-L reaches 64.9 and 76.2 after 10 and 100 epochs, respectively. To further evaluate robustness under challenging conditions, three degraded data subsets are generated. Building on the strengths of convolutional and transformer-based detectors, this work introduces an image-based selection model that selects the most suitable detector for each input image. Experimental results demonstrate substantial zero-shot degradation, strong recovery after fine-tuning, and consistent performance improvements under degraded conditions using the proposed selection strategy. Our method achieves gains of up to 4 𝑚𝐴𝑃 points over the best standalone detector without incurring the computational overhead. The proposed framework provides a context-aware and computationally efficient perception enhancement strategy suitable for real-world autonomous driving systems.
Ahmed M. Reda, Naser El Sheimy, Adel Moussa· The International Archives o...· 0 citations
Real-time perception is a foundational requirement for advanced driver assistance systems (ADAS) and autonomous vehicles, yet embedded automotive platforms impose severe constraints on compute, memory, and power. This paper presents an optimized semantic segmentation architecture derived from the RetinaNet detection framework, adapted for dense pixel-wise prediction and tailored for deployment on resource-constrained embedded hardware. The proposed architecture, termed Opt-RetinaSeg, replaces the standard ResNet-50 backbone with a hybrid lightweight feature extractor, restructures the Feature Pyramid Network (FPN) to reduce redundant multi-scale computation, and introduces a compact segmentation head guided by focal-loss-inspired class balancing to address the severe foreground-background imbalance common in road scenes. We further apply a three-stage optimization pipeline consisting of structured channel pruning, post-training INT8 quantization, and knowledge distillation from a high-capacity teacher network. Evaluated on the Cityscapes and BDD100K datasets and deployed on an NVIDIA Jetson Xavier NX and a Qualcomm QCS610 automotive SoC, the proposed model achieves 73.9% mIoU at 70.4 FPS, representing a 7.4x inference speedup and a 4x reduction in model size relative to the ResNet-50 baseline, with less than 3% accuracy degradation. These results indicate that RetinaNet-derived architectures, when systematically optimized, are viable candidates for real-time semantic segmentation in embedded automotive perception pipelines
This work addresses major challenges in sensor fusion, including data synchronization, coordinate transformation, real-time computation and conflict resolution of heterogeneous sensor data.
Yu-He Lu· Applied and Computational En...· 0 citations
: Visual simultaneous localization and mapping (VSLAM) is a key technology for mobile robotics, autonomous driving, and embodied intelligence, enabling self-localization, environment reconstruction, and scene understanding. Although conventional geometric methods have achieved notable success, their performance often degrades in challenging conditions, such as low-texture scenes, severe illumination changes, dynamic interference, and long-term environmental variations. Recent advances in deep learning have created new opportunities to improve VSLAM through stronger feature representations, learned priors, semantic perception, and emerging map representations. At the same time, the increasing adoption of learning-based modules has raised important questions about integration strategies, generalization, interpretability, and real-time deployment. This paper presents a systematic review of deep learning-enhanced VSLAM, with a particular focus on how learning models are incorporated into classical simultaneous localization and mapping (SLAM) pipelines and how they function within the overall system. To provide a unified perspective, existing methods are organized into five categories according to their fusion interfaces with geometric SLAM pipelines: observation-level interfaces, constraint/prior/weight-level interfaces, solver-level interfaces, representation-level interfaces, and system-level integration interfaces. Based on this taxonomy, representative approaches are comparatively analyzed for accuracy, robustness, efficiency, and deployability. In addition, this review summarizes common design principles, including geometric consistency constraints, error propagation characteristics, and typical failure modes, and further discusses open challenges and future directions such as lightweight deployment, cross-domain adaptation, dynamic map modeling, and long-term consistency maintenance. This review aims to provide a structured reference for the analysis, design, and deployment of learning-enhanced VSLAM systems.
Xiruo Chen, Qi Ouyang, Sihong Meng et al.· Computers, Materials & C...· 0 citations
Recent research in machine and deep learning has shown the potential of learningbased motion planning approaches to improve the driving behavior of automated vehicles, especially in complex environments. However, their complex nature and lack of transparency can hinder explainability and trustworthiness and complicate safety assurance. Motivated by these challenges, we propose a hybrid planning architecture that combines the advantages of machine learning with the verifiability and the determinism of classical approaches. Specifically, we developed a deep neural network to interpret complex traffic scenes and propose driving behavior, while an optimization-based supervision layer validates this proposal and enforces explicit drivability and safety constraints. We evaluate the learned planner's driving behavior in open-loop studies on real-world urban data, discuss system integration aspects for stable closed-loop operation, and report results from real-world deployment on our research vehicle karl..
Jean-Pierre Busch, Guido Linden, Jan Bergmann et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
