Deep Reinforcement Learning-Based Joint Control for Rotatable-Array UAV Transportation Communications

Abstract

Future transportation networks may require aerial communication platforms capable of providing flexible and reliable services to vehicular terminals. In conventional unmanned aerial vehicle (UAV) communication systems, the antenna geometry is commonly treated as fixed, which limits the attainable directional gain when the relative geometry between the UAV and users changes significantly. This work considered a UAV equipped with a mechanically reconfigurable antenna array and studied its joint motion and transmission control under finite-blocklength communication. A sequential optimization problem was formulated to maximize the accumulated user throughput by jointly optimizing the UAV trajectory, the array orientations, and the transmit beamforming vectors, subject to the UAV kinematic constraints, the UPA orientation constraints, and the transmission energy budget. The resulting problem involves nonlinear coupling among platform motion, antenna pointing, beamforming, and finite-blocklength rate expressions, making conventional optimization computationally demanding. To obtain an adaptive control policy, a soft actor–critic-based deep reinforcement learning method was developed. The simulation results showed that jointly controlling the UAV mobility, array orientation, and beamforming improves the achievable finite-blocklength transmission performance compared with benchmark schemes, demonstrating the effectiveness of the proposed framework in enhancing reliable data delivery for UAV-assisted transportation infrastructure applications.