MAFRL: A Multi-Agent Flow-Balance Reinforcement Learning Framework for Resource Allocation in LEO Satellite Systems

Abstract

Low Earth orbit (LEO) satellite systems are expected to deliver ubiquitous broadband connectivity, but their dynamic topology and limited on-board resources challenge beam hopping (BH) scheduling and inter-satellite load balancing. This paper proposes a multi-satellite cooperative BH algorithm based on multi-agent flow-balanced reinforcement learning (MAFRL). MAFRL formulates cooperative BH as trajectory-level distribution matching and uses a centralized partition function network during training to guide decentralized beam-level actors. Its reward design jointly captures throughput, time-sensitive traffic delay, and maximum and average inter-satellite load gaps in overlapping coverage areas. Simulations show that MAFRL reduces the maximum load gap by up to 18.2% and the average load gap by up to 27.5% compared with baselines, while improving throughput and delay under high traffic demands.