2026· IEEE Transactions on Cognitive Communications and Networking· Vol 12, pp. 10932-10947· 0 citations· 31 references
Abstract
Low Earth Orbit (LEO) constellations are required to process increasing volumes of heterogeneous tasks from ground networks. Intermittent inter-satellite links, heterogeneous onboard resources, and time-varying traffic loads make collaborative task offloading difficult for static or reactive strategies. Although multi-agent reinforcement learning (MARL) provides an adaptive solution, existing model-free MARL methods often suffer from slow convergence, insufficient foresight, and limited robustness in dynamic satellite environments. To address these challenges, this paper proposes a World Model-Based Multi-Agent Proximal Policy Optimization (WM-MAPPO) framework for space computing power networks. The offloading problem is formulated as a partially observable multi-agent decision-making process, where LEO satellites make decentralized decisions under incomplete local observations. A predictive world model learns latent transition dynamics of network states and provides future context for proactive planning. Meanwhile, a Transformer-based policy architecture captures inter-agent dependencies and supports cooperative scheduling under centralized training and decentralized execution (CTDE). Simulation results show that WM-MAPPO achieves higher task completion ratios, lower average latency, improved energy efficiency, and stronger robustness than model-free MARL baselines, heuristic methods, Lyapunov-based scheduling, and MINLP-inspired optimization.
This paper addresses the joint task offloading and resource allocation problem in multi-user MEC systems and proposes a decentralized control framework based on Multi-Agent Reinforcement Learning (MARL), which achieves lower total system cost and faster convergence than the full-local, full-offload, and heuristic baselines.
Youssef Oukissou, Mohamed Amine Meddaoui, Ayoub Belaidi et al.· International journal of Com...· 0 citations
Future sixth-generation non-terrestrial networks are expected to combine low Earth orbit (LEO), medium Earth orbit (MEO), and geostationary Earth orbit (GEO) satellites, whose complementary layers must be coordinated through joint user association, power allocation, and handover management under fast LEO dynamics. This paper studies this problem by formulating it as a mixed-integer nonlinear program and decomposing it into a multi-agent reinforcement learning (MARL) policy that selects the associations and a convex power-allocation subproblem solved exactly at each time slot that defines the reward of the MARL part. The association policy is trained with multi-agent proximal policy optimization (MAPPO) and the targeted multi-agent communication (TarMAC) mechanism, and is made aware of the orbital layer through a state that encodes layer-dependent handover penalties. Evaluated on a realistic multi-constellation scenario built from real two-line element data over Nairobi, Kenya, the proposed policy reaches 92% of the throughput of a greedy signal-to-noise ratio (SNR) maximizing scheme while triggering more than four times fewer handovers, and improves throughput by roughly 14% over a conservative stay heuristic. Compared to an LEO-only learned policy of identical architecture, it attains slightly higher throughput with fewer handovers by offloading a fraction of the users to the MEO and GEO layers, an emergent multi-orbit behavior that drives its favorable throughput and handover trade-off.
Yassine Afif, Ashutosh Balakrishnan, Philippe Martins et al.· 0 citations
A constraint-aware multi-agent edge collaborative offloading algorithm (CARE-CTDE) that achieves better scheduling performance, resource utilization, and constraint satisfaction than baseline methods in dynamic heterogeneous MEC scenarios, demonstrating its effectiveness and robustness for constrained edge computing systems.
Yuxuan Yang, Hexing Wang, Yang Zhou· Mathematics· 0 citations
Sixth-generation (6G) networks are expected to rely on agentic artificial intelligence for zero-touch, self-managed orchestration of heterogeneous network slices serving enhanced mobile broadband (eMBB), ultra-reliable low-latency communication (URLLC), and massive machine-type communication (mMTC). A central and under-studied challenge for adaptive multi-agent resource management (AMRM) in such settings is multi-timescale non-stationarity: channel fading evolves per time-slot, user demand shifts at the window scale, and service-level agreement (SLA) regimes change at an operational scale. Single-timescale multi-agent reinforcement learning (MARL) algorithms cannot track all three signals cleanly—a learning rate fast enough for the per-slot channel destabilises the coordination structure that governs longer-timescale policies. This paper proposes Nested-MARL, an independent-learner actor-critic algorithm in which each agent’s parameters are partitioned into three groups updated at separated rates <inline-formula> <tex-math notation="LaTeX">$\alpha _{0}\!\ll \!\alpha _{1}\!\ll \!\alpha _{2}$ </tex-math></inline-formula>, with a continuum-memory exponential moving average (EMA) anchoring the slowest group. The design is grounded in the Nested Learning paradigm of Behrouz et al. (2025) and is extended here from single-model continual learning to decentralised multi-agent coordination. We establish a finite-time convergence result in the two-timescale stochastic approximation framework showing that under standard regularity and timescale-separation conditions, Nested-MARL achieves <inline-formula> <tex-math notation="LaTeX">$O(T^{-1/2})$ </tex-math></inline-formula> fast-group convergence vs. an <inline-formula> <tex-math notation="LaTeX">$\Omega (T^{-1/3})$ </tex-math></inline-formula> lower bound for any single-timescale algorithm. An empirical study on a three-agent 6G slicing simulator with continuous multi-timescale drift shows Nested-MARL outperforms independent PPO (IPPO) in mean reward at every drift severity we test (<inline-formula> <tex-math notation="LaTeX">$\kappa \!\in \!\{0.5,1.0,1.5,2.0\}$ </tex-math></inline-formula>) and by + 8.6% in sample efficiency over the first 40 episodes at <inline-formula> <tex-math notation="LaTeX">$\kappa {=}1.5$ </tex-math></inline-formula> (<inline-formula> <tex-math notation="LaTeX">$n{=}10$ </tex-math></inline-formula> seeds, <inline-formula> <tex-math notation="LaTeX">$p\lt 0.05$ </tex-math></inline-formula>). A controlled ablation establishes that stripping timescale separation reduces performance below the IPPO baseline, isolating timescale separation as the causal mechanism. Nested-MARL also reduces policy switching cost by 16.6%, an operationally meaningful benefit for zero-touch orchestration. The complete simulator, agents, and 60 + per-seed training runs are released as open source.
Abraheem Rashid, Faisal Iradat, Waseem Iqbal et al.· IEEE Open Journal of the Com...· 0 citations
Modern grid operation is increasingly a sequential collaborative control problem under renewable uncertainty, storage dynamics, flexible demand, transmission coupling, and carbon-aware corrective redispatch. This paper focuses on sub-hourly dynamic OPF assistance and safe regional redispatch after forecast updates or emergent network stress. We formulate networked grid operation as a constrained decentralized partially observable Markov decision process and propose a safe multi-agent collaborative learning framework. The method integrates dynamic transfer-stress tracing, a dual-view graph encoder over the physical grid and a real-time stress graph, consensus-based dual coordination for globally coupled constraints, and a differentiable safety projection that maps tentative decisions to executable actions. The framework aims to reduce operating cost, load shedding, renewable curtailment, and carbon-relevant corrective burden, while maintaining feasibility with respect to local linearized surrogate limits and empirically reducing violations in the full simulator. Experiments on chronological grid benchmarks evaluate comparison, ablation, robustness, efficiency, statistical evidence, visualization, and generalization, showing improved trade-offs among cost, reliability, safety, and renewable accommodation.
Jiayi Zhang, Bing Fang, Huanxiu Xiao et al.· International journal of pat...· 0 citations
This paper proposes a multi-agent reinforcement learning (MARL) framework for TSN scheduling, where each TSN queue is modeled as an autonomous agent and the Heterogeneous-Agent Proximal Policy Optimization (HAPPO) algorithm is employed to explicitly model inter-agent dependencies and jointly optimize service delivery across queues.
Marcos Carvalho, Fatih Temiz, Shavbo Salehi et al.· 0 citations