Skip to content

A Heterogeneous Multiagent Reinforcement Learning Approach for Robust Uplink Beamforming in Maritime Satellite Communications

2026 · IEEE Transactions on Aerospace and Electronic Systems · Vol 62, pp. 14229-14243 · 0 citations · 34 references
Computer Science

Abstract

Maritime satellite communications (SATCOMs) are expected to support high-capacity ship-to-satellite uplinks for remote maritime services beyond terrestrial coverage, with low-Earth-orbit (LEO) satellites providing wide-area connectivity. However, robust uplink beamforming in LEO maritime SATCOMs is challenging because dynamic ship–satellite geometry, wave-induced attitude motion, imperfect channel state information, and multiship interference make transmit power, ship-side transmit beamforming, and satellite-side receive combining tightly coupled. Accordingly, we formulate a long-term spectral efficiency (SE) maximization problem under transmit-power and quality-of-service constraints. An attitude-aware uplink channel model is developed by incorporating roll, pitch, and yaw motions into the effective angle-of-departure/angle-of-arrival evolution. Based on this model, the problem is cast as a heterogeneous decentralized partially observable Markov decision process. We then propose a robust heterogeneous cooperative QMIX (RHC-QMIX) framework under centralized training and decentralized execution, where type-specific recurrent local Q-networks, history-refined angular features, and centralized monotonic value mixing coordinate ship and satellite agents. Extensive simulations demonstrate that in the load-controlled scalability evaluation, RHC-QMIX achieves an average network SE of 25.32 bps/Hz, improves over alternating optimization by up to 51.00% as the satellite load increases, and outperforms heterogeneous cooperative QMIX by 16.38% on average under network-size scaling; it also maintains more stable SE under severe sea-state-induced ship motion.

View source

Similar papers

Preprint Aug 2026

Multi-Agent Reinforcement Learning for Joint Handover Management and Power Allocation in Multi-Orbit Satellite Networks

The proposed multi-agent reinforcement learning policy attains slightly higher throughput with fewer handovers by offloading a fraction of the users to the MEO and GEO layers, an emergent multi-orbit behavior that drives its favorable throughput and handover trade-off.

Yassine Afif, Ashutosh Balakrishnan, Philippe Martins et al. · 0 citations
Open access 2026

LLM-Guided Multi-Agent Joint Velocity and Spectrum Optimization in Advanced Air Mobility

A Large Language Model-guided cooperative decision-making framework for joint velocity control and bidirectional channel selection in an AAM system with Aerial Vehicles communicating with ground Base Stations while following predefined linear routes is proposed.

Qingyang Li, Adnan Quadri, Hongxiang Li et al. · 0 citations
2026

A Reinforcement Learning-Based Scheduling Scheme for FSO and RF Hybrid Satellite-to-Ground Transmission Systems

Low Earth orbit (LEO) satellite-terrestrial communication systems grapple with significant challenges posed by their inherent dynamism and substantial transmission delays. To address these critical issues, this paper proposes a novel hybrid-medium transmission optimization framework that leverages high-altitude platforms (HAPs) as relays. Our primary objective is to minimize end-to-end system delay through the joint optimization of transmission mode selection and wireless communication resource allocation. The resulting joint optimization problem is formulated as a computationally intractable mixed-integer nonlinear programming (MINLP). We present a hierarchical solution strategy to tackle this complexity. Firstly, Lagrangian optimization is employed to analytically derive the intrinsic coupling between resource allocation and transmission mode selection, thereby simplifying the problem into a sequential decision-making process. This sequential problem is subsequently framed as a Markov decision process (MDP), enabling the design of a deep reinforcement learning (DRL) agent tasked with dynamically learning the optimal transmission mode selection policy. By maximizing cumulative long-term rewards, our DRL-based approach effectively reduces overall system delay, unlocking enhanced performance potential for future 6G networks.

Yi Huang, Jin Li, Yanwen Zhu et al. · 0 citations
Preprint Sep 2026

Rotatable Antenna Enabled Multi-Satellite Communications: Joint Satellite Selection and Boresight Trajectory Optimization

This paper considers a satellite-to-ground communication system in which a ground station (GS) equipped with independently rotatable antenna (RA) elements jointly decodes independent streams from multiple low-Earth-orbit (LEO) satellites over a shared time--frequency resource. Specifically, we formulate a two-timescale throughput maximization problem under exogenous cochannel interference, capturing serving-set composition, RA-enabled channel shaping, time-varying satellite geometry, and mechanically constrained inter-epoch reconfiguration. We first characterize the joint effects of interference-whitened channel strength and spatial separability on multi-satellite reception, motivating the joint design of satellite selection and RA control. With the RA trajectory fixed, we establish the monotone submodularity of the epoch-level selection objective and construct an incumbent-tight modular lower-bound surrogate, leading to an efficient discrete Minorization-Maximization (MM) selection algorithm. For fixed serving sets, we develop slew-feasible RA updates based on Riemannian gradients and organize them into a two-color parallel update scheme. The two blocks are integrated into a monotone alternating algorithm with guaranteed objective convergence. Simulations demonstrate consistent gains over benchmark schemes and reveal an optimal balance between channel strength and spatial separability. The results further show that satellite selection is particularly important in underloaded and actuator-limited regimes, whereas RA shaping becomes more influential near full spatial loading.

Xingxiang Peng, Qingqing Wu, Hai-Ying Hu et al. · 0 citations
2026

Distributed Cooperative Beamforming for Spectrum Sharing in GEO-LEO Heterogeneous Multi-Satellite System

Due to their resilience and global coverage, satellite networks are poised to become a key component for non-terrestrial networks in the future. However, given the scarcity of spectrum resources, the dense deployment of low Earth orbit (LEO) satellites introduces significant interference challenges. Meanwhile, the limited computing power and backhaul capacity of satellites have become bottlenecks hindering the development of advanced interference mitigation techniques. This paper studies beamforming in GEO-LEO heterogeneous multi-satellite systems. For the GEO system, we develop a multicast beamforming approach based on a nonlinear eigenvalue problem (NEPv) for beam direction design and Lagrange dual decomposition (LDD) for power allocation. For the LEO system, we propose a general distributed beamforming framework and two distributed beamforming methods. Specifically, we first leverage equivalent multi-dimensional fractional programming (FP) to decompose the objective function. The resulting subproblems are then optimized in a distributed manner across multiple satellites via the parallel block coordinate descent (PBCD) method. For the distributed optimization subproblems, we derive semi-closed-form solutions using Lagrangian dual ascent (LDA) and alternating direction method of multipliers (ADMM) for scenarios without and with GEO-LEO interference avoidance, respectively. Simulation results show that the proposed NEPv-LDD method strictly satisfies the QoS constraints of users and achieves near-optimal performance with low complexity. For the LEO beamforming, the developed distributed FP (DiFP) framework exhibits strong scalability in large-scale constellations. Built upon the DiFP framework, the proposed DiFP-NoSIA incurs almost no performance loss, while DiFP-ADMM shows only an 8.58% performance degradation compared to the centralized benchmark.

Xin Chen, Zhiyong Luo · 0 citations
Open access 2026

Multi-RIS-Assisted Satellite Compact Ultra-Massive Antenna Array for Massive Uplink Transmission

High-capacity satellite network is the cornerstone of future space-air-ground integrated networks. However, the satellite uplink transmissions still face critical challenges, including severe path loss, complex multi-user interference, and payload constraints. Recently, Reconfigurable Intelligent Surfaces (RIS) and Fluid Antenna Systems (FAS) have shown promise for satellite communications through their dynamic signal reconfiguration. This paper proposes a multi-RIS-assisted satellite Compact Ultra-Massive Antenna Array (CUMA) architecture for multi-user satellite uplink transmission. Specifically, we deploy multiple RISs on the terrestrial side to separate interfering Line-of-Sight (LoS) channels via optimized phase shifts, and adopt a CUMA receiver on the satellite to further mitigate interference through FAS port selection. To solve a sum-rate maximization problem, we alternately optimize FAS port selection using a Forward-Backward Greedy Selection (FBGS) algorithm and RIS phase shifts based on Fractional Programming (FP). To the best of our knowledge, this is the first work to jointly optimize multi-RIS and CUMA in a satellite uplink context, where strong LoS and extreme path loss fundamentally distinguish the design from terrestrial counterparts. Simulation results confirm the effectiveness of the proposed architecture across frequency bands. At 6 GHz, our scheme achieves 181% and 32% rate gains over fixed antennas and traditional CUMA schemes, respectively, while the gains also reach 138% and 27% at 26 GHz, illustrating superiority in both interference-limited and noise-limited regimes.

Kai Feng, Runke Fan, Tianheng Xu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.