Short-Term Temporal Transformer Deep Reinforcement Learning with Dynamic-Window-Constrained Actions for LiDAR-Based Local Navigation

Abstract

Local navigation in dynamic indoor environments is challenging, as a single 2-D LiDAR scan contains no obstacle motion information, and control commands must satisfy the robot’s velocity and acceleration constraints. This paper proposes ST-DRL, a value-based local planner combining a lightweight temporal Transformer encoder with a dynamic-window-constrained action interface. The encoder extracts motion cues from H = 4 sectorized LiDAR scans and goal-relative ego states without object tracking or velocity labels. Double DQN picks from 25 discrete action indices, which are mapped online to velocity pairs within the dynamic window so that the prescribed command-space velocity and acceleration bounds are satisfied by construction. Evaluated over 900 randomized episodes per method, i.e., 9000 episodes in total across ten methods and six obstacle densities (0.37–2.24 obstacles/m2 with half moving), ST-DRL achieves a 91.56% success rate and an 8.44% collision rate, surpassing tuned DWA, LSTM-based Double DQN, PPO, and dynamic-window-projected PPO (PPO-DW) by 45.6, 28.7, 24.9, and 15.1 percentage points, respectively. Ablations verify key module contributions. ST-DRL runs in 2.01 ms in simulation and 4.8 ms on Jetson Nano. Real Ackermann UGV tests are consistent with the simulation trend, enabling preliminary deployment feasibility without domain randomization under tested conditions, while formal collision and Ackermann feasibility guarantees are not provided. The real-vehicle trials should be interpreted as a deployment-feasibility study rather than a statistically powered benchmark. They involve one Ackermann UGV, one indoor track, two moving-obstacle settings, and 20 trials per setting. Therefore, the results do not establish robust generalization across different buildings, obstacle behaviors, robot speeds, LiDAR models, or vehicle platforms.