AGTA: Topology-Aware Sequential Decision-Making in Multi-Agent Reinforcement Learning
TL;DR
Action Generation with Topology Awareness (AGTA), a topology-aware sequential decision-making framework in MARL that integrates inter-agent correlation modeling with topology-guided decision-order optimization, and outperforms the state-of-the-art counterparts.
Abstract
Multi-agent reinforcement learning (MARL) has long grappled with fundamental coordination challenges due to the existence of complex inter-agent correlations that are inherent in multi-agent systems. While the recent advancement of the sequential decision-making paradigm provides fine-grained supervision for the multi-agent decision-making process, the absence of computationally tractable solutions for inter-agent correlation management remains a critical challenge and substantially constrains the impact of this paradigm. To tackle this challenge, in this paper, we introduce Action Generation with Topology Awareness (AGTA), a topology-aware sequential decision-making framework in MARL that integrates inter-agent correlation modeling with topology-guided decision-order optimization. AGTA extracts inter-agent mutual attention via multi-agent transformer during the learning dynamics. Subsequently, it captures directed acyclic graphs(DAGs) directly from the extracted attention matrices to model inter-agent correlations. Finally, it refines the action generation order by analyzing and solving topological constraints, thus realizing inter-agent correlation management. Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art counterparts.