Decentralized Multi-Agent Reinforcement Learning with Communication for Uncertain Environments

Abstract

This paper presents a novel decentralized multi-agent reinforcement learning (MARL) algorithm designed to address the challenges posed by environmental uncertainty and information asymmetry in multi-agent systems. The core of the algorithm lies in leveraging communication between agents, inspired by approaches like MADDPG, to facilitate information sharing and collaborative learning. We demonstrate that this communication-based strategy significantly enhances the performance of multi-agent systems, particularly when dealing with stochastic environments where individual agent observations are incomplete and unreliable. The algorithm incorporates a novel uncertainty estimation module to dynamically adjust the communication frequency and content, optimizing for efficiency and robustness. We formalize the problem as a Partially Observable Markov Decision Process (POMDP) and outline the key components of the proposed solution, focusing on the decentralized training and execution strategy. Experimental results (simulated) on benchmark MARL environments illustrate the effectiveness of the proposed method compared to traditional centralized and decentralized approaches. The key contributions of this work are a communication-augmented MARL framework that addresses uncertainty and information asymmetry and a practical approach for decentralized learning and execution in complex, dynamic environments.