Reinforcement learning-guided multi-objective trajectory planning for obstacle avoidance in robotic manipulators

Abstract

Robotic manipulators operating in cluttered environments require collision-free trajectories that remain executable under kinematic and dynamic constraints. This paper proposes a reinforcement learning (RL)-guided multi-objective trajectory planning framework, termed RL-MOP-HNE, for a 6-DOF UR5 manipulator. The planning model simultaneously minimizes path length, energy consumption, and execution time while satisfying collision-avoidance, kinematic, and dynamic constraints. A tabular SARSA agent is embedded into the evolutionary search to adaptively select search behaviours according to the current optimization state. To improve the balance between exploration and exploitation, a Gaussian-perturbation adaptive hybrid crossover operator is integrated with a hierarchical neighborhood evolution (HNE) strategy, enabling progressive population refinement throughout the search process. The proposed method is evaluated in three representative environments with increasing planning complexity, including single-obstacle, narrow three-obstacle, and irregular five-obstacle scenarios, and is compared with MOEA/D, MOPSO, MSCLPSO, NSGA-II, and RL-NSGA-II. Experimental results show that RL-MOP-HNE generates feasible trajectories in all test cases and achieves the lowest dynamic-stability-prioritized composite scores among the compared algorithms. The planned trajectories exhibit smoother joint motion and lower velocity fluctuations, although these improvements are generally accompanied by longer execution times. Complementary analyses, including time scaling, manipulability, clearance evaluation, statistical significance tests, and ablation studies, further explain the performance characteristics of the proposed framework and quantify the contribution of its key components. The proposed framework is therefore well suited to robotic applications where motion stability and dynamic executability are of greater importance than minimum-time operation.