Multimodal Digital-Twin Simulations for Off-Road Autonomous Vehicles
TL;DR
A large-scale simulation platform for off-road autonomous driving generated in the Unity3D engine that facilitates progress for systematic evaluation of learning-based autonomous driving models in unstructured, safety-critical environments.
Abstract
Research in autonomous driving has largely focused on structured, on-road environments in densely populated urban areas, leaving off-road autonomy comparatively underexplored despite its importance for applications such as search and rescue, agriculture, and defense. Progress in developing solutions for off-road autonomy is heavily constrained by the high cost, safety risks, and limited coverage of real-world off-road data collection, particularly for rare terrain-induced failure cases. To address these challenges, we introduce a large-scale simulation platform for off-road autonomous driving generated in the Unity3D engine. Our simulator provides indefinite data from multiple RGB cameras, LiDAR, GPS, and IMU sensors to support research in long-duration applications. Our platform also facilitates research in risk-aware planning through dynamic weather, lighting, and annotations for sequence-level failures like collisions and weather-induced sensor blockages. By combining long-duration temporal coverage, multimodal sensing, and procedurally generated terrain diversity, our simulator facilitates progress for systematic evaluation of learning-based autonomous driving models in unstructured, safety-critical environments.