Multi-Agent Reinforcement Learning for Handover Management and Power Allocation in LEO Satellite Networks

Abstract

Due to their high orbital velocity, low Earth orbit (LEO) satellites traverse overhead in minutes, forcing ground users to frequently switch their serving beam or satellite. Such handover disrupts ongoing transmissions, introduces signalling overheads and synchronization delays across the constellation, and reduces the effective throughput perceived by ground users. To mitigate this, we consider a joint beam assignment and power allocation problem for a multi-beam LEO downlink. The resulting problem is a mixed-integer non-linear program, which in general is hard to solve directly. We decompose it into a convex power allocation and association subproblems. The power allocation subproblem is solved to global optimality at each time slot, while the association subproblem is solved via multi-agent reinforcement learning (MARL) policies. Namely, we propose multi-agent proximal policy optimization (MAPPO) with two communication variants: (i) using explicit hand-crafted neighbour features (MAPPO-<inline-formula> <tex-math notation="LaTeX">$k_{n}$ </tex-math></inline-formula>) and (ii) using learned attention-based messages, with the targeted multi-agent communication (TarMAC) method. For TarMAC, each agent coordinates their beam selection by exchanging messages with their geographic neighbours. Numerical analysis conducted on real Starlink two-line element (TLE) data shows that all reinforcement learning (RL) policies reduce handovers by at least 94% relative to a <monospace>Greedy</monospace> baseline that always selects the highest-signal to noise ratio (SNR) beam. It also shows that these policies improve mean effective rates by up to 30% over a static policy that keeps users on their current beam as long as the satellite remains visible. The two communication paradigms define complementary operating schemes depending on the operator’s objective. MAPPO-<inline-formula> <tex-math notation="LaTeX">$k_{n}$ </tex-math></inline-formula> improves throughput at the cost of more handovers, while TarMAC-<inline-formula> <tex-math notation="LaTeX">$k_{n}$ </tex-math></inline-formula> simultaneously improves rate and reduces handovers relative to proximal policy optimization (PPO), offering a better trade-off when handover cost is a primary concern.