Joint Trajectory and User Association Optimization in HAP-UAV Integrated Networks Based on MADQN-DPSO
Abstract
HAP-UAV integrated networks are pivotal for 6G ubiquitous coverage and massive connectivity. While aerial nodes, such as uncrewed aerial vehicles (UAVs) and high altitude platforms (HAPs), extend coverage beyond terrestrial base stations, they also introduce resource management and trajectory optimization challenges due to dynamic user distributions and inter-UAV coverage overlap. Conventional single-agent deep reinforcement learning and heuristic association rules struggle with the NP-hard user association problem and the curse of dimensionality in hybrid action spaces; they tend to prematurely converge to poor local optima, lacking the ability to continuously refine their solutions. To this end, this paper proposes a hybrid framework that integrates multi-agent deep Q-networks (MADQN) with discrete particle swarm optimization (DPSO). The framework decomposes the joint problem into two coupled subproblems. For user association, an enhanced DPSO with a constraint handling efficiently explores the discrete search space to produce high-quality feasible assignments. For trajectory planning, MADQN incorporates a self-attention mechanism that dynamically aggregates state information from neighboring UAVs, capturing complex inter-agent dependencies and enabling implicit communication. This mechanism enhances decision-making robustness and enables agents to adaptively refine their policies through coordinated exploration, avoiding premature convergence to inferior solutions. By integrating discrete combinatorial optimization with multi-agent discrete-action control over a trajectory space, the proposed framework effectively overcomes the curse of dimensionality and yields high-quality solutions. Simulation results demonstrate that the proposed scheme achieves significant improvements in total system rate across various user densities and UAV scales compared with some typical benchmarks.