Empirical Evaluation of Proximal Policy Optimization for UAV Navigation in Constrained Indoor Environments

Abstract

Autonomous navigation of unmanned aerial vehicles (UAVs) in constrained indoor environments remains a challenging problem due to limited maneuvering space and high collision risk. This paper presents an empirical evaluation of a reinforcement learning-based approach for UAV path planning using Proximal Policy Optimization (PPO) in a high-fidelity AirSim simulation environment. The UAV operates in a cluttered warehouse setting with static obstacles, where it must reach predefined goal locations while minimizing collisions and trajectory inefficiencies. We conducted extensive experiments to analyze the reward function shaping, training dynamics, hyperparameter sensitivity, and navigation performance across multiple goal configurations. Results show that the PPO-based agent achieves an overall success rate of (78%) with stable convergence and improved path efficiency, while maintaining low collision rates in less constrained regions. However, performance degrades in obstacledense areas requiring sharp maneuvers, highlighting limitations in generalization. The study provides insights into the impact of reward design and environment complexity on reinforcement learning performance for indoor UAV navigation.