A Reinforcement Learning-Based Multi-Strategy Differential Evolution Algorithm for Agricultural UAV Path Planning

Abstract

In the realm of precision agriculture, agricultural UAV path planning is challenging because the UAV must avoid obstacles, follow uneven terrain, and satisfy multiple flight constraints simultaneously. Differential evolution (DE) has been widely adopted for this problem because of its simple structure and effective optimization capability. However, existing DE-based methods often become trapped in local optima and cannot effectively balance exploration and exploitation in complex search environments. To address these issues, this paper proposes a reinforcement learning-based multi-strategy differential evolution algorithm, named PPOMSDE. By introducing Proximal Policy Optimization (PPO) to construct a multi-dimensional state pool and an action pool, PPOMSDE enables adaptive strategies for individuals, improving strategy selection during the search process. An independent multi-buffer is adopted to ensure strict data isolation and efficient learning to avoid strategy confusion. In addition, an adaptive triplet mechanism which partitions the population into fitness-based tiers (best, medium, and worst) assigns different control parameters and mutation strategies to individuals with different fitness levels, improving the balance between global exploration and local exploitation. Extensive experiments on the CEC’2014 and CEC’2017 benchmark suites demonstrate the effectiveness of PPOMSDE. The proposed method achieves the lowest average performance ranks of 1.39 on the combined 10-D and 30-D CEC’2014 benchmarks and 1.03 on the 10-D CEC’2017 benchmarks. In agricultural UAV path planning, PPOMSDE generates safer and smoother flight paths while maintaining accurate terrain-following flight, reducing the overall cost by an average of 22.42% compared with ISDE, L-SHADE, SHADE, and ISHACDE.