USV Collision Avoidance via DRL With Spatiotemporal Expert Guidance: From Threat Attention to Safety Shield

Abstract

Deep reinforcement learning (DRL) provides an end-to-end approach to collision avoidance for uncrewed surface vehicles (USVs). However, it typically exhibits inadequate threat perception capabilities and low sample efficiency in complex environments. To address these challenges, a DRL-based framework that couples beam-wise attention learning with velocity obstacle (VO) imitation is developed. A threat-aware attention module tailored to the beam map is designed to achieve direction-level threat allocation, enabling the policy to focus on high-threat regions. VO is introduced to serve as an expert policy that provides spatiotemporally informed action demonstrations during both pre-training and online training phases, thereby improving training efficiency and safety. A VO-inspired safety shield is developed that filters policy commands at execution time, improving robustness under both command and execution disturbances. Experimental results demonstrate that our method achieves a success rate of approximately 92% and shows improved training efficiency compared with the baselines. Field experiments validate its preliminary practical feasibility. Note to Practitioners—This work aims to address collision avoidance for USVs in multi-obstacle environments. Existing methods require extensive tuning in complex interactions, while DRL often incurs high training cost and involves risky exploration. Beam maps are isomorphic to LiDAR range measurements and are easy to integrate in practice, yet threat-aware solutions tailored to this representation remain limited. A framework is proposed that jointly leverages a learned policy and expert-guided training, improving training efficiency and safety under beam-map perception inputs. The proposed method has been validated in both simulation and field experiments.