RAO++: Realistic Real-time Multi-vehicle Collaboration on Asynchronous Sensors
TL;DR
RAO++ is presented, a real-time cooperative perception system that merges asynchronous sensor data from different vehicles through the novel designs of motion-compensated occupancy flow prediction, on-demand data sharing, with a variety of system optimizations to improve the accuracy and coverage of the perception system.
Abstract
Cooperative perception enables connected autonomous vehicles to extend their sensing range and overcome occlusions by exchanging sensor data. However, its real-world deployment is hindered by asynchronous sensor streams and inaccurate localization of occluded regions. This work presents RAO++, a real-time cooperative perception system that merges asynchronous sensor data from different vehicles through our novel designs of motion-compensated occupancy flow prediction, on-demand data sharing, with a variety of system optimizations to improve the accuracy and coverage of the perception system. Our comprehensive evaluation, including real-world and emulation experiments under diverse LiDAR configurations, shows that RAO++ outperforms asynchronous-unaware methods by more than 34% in perception coverage and by up to 14% in perception accuracy. Moreover, RAO++ reduces latency by 1.2–3.5× and communication overhead by 45–70% compared with the state-of-the-art asynchronous-aware baseline, while maintaining comparable detection accuracy. Finally, RAO++ demonstrates a practical data overhead of 12.8 KB per frame, enabling deployment under realistic bandwidth constraints.