Decentralized Federated Reinforcement Learning with Byzantine Agreement

Abstract

Decentralized Federated Reinforcement Learning (DFRL) presents a promising approach to training robust and adaptable RL agents by leveraging distributed data and computational resources. However, this paradigm is susceptible to attacks from Byzantine actors who can inject malicious updates, jeopardizing the learning process and potentially leading to catastrophic outcomes. This paper introduces a novel framework for DFRL that incorporates Byzantine agreement protocols to mitigate these vulnerabilities. Our approach enables agents to collaboratively learn while simultaneously resisting manipulation and ensuring convergence. We formalize the problem, define the key components of the system, and present a theoretical analysis demonstrating the effectiveness of our method. The core claim is that training RL agents across multiple devices introduces vulnerabilities to malicious actors. The core mechanism applies Byzantine agreement protocols to decentralized federated RL, enabling agents to learn collaboratively while resisting manipulation and ensuring convergence. This work significantly advances the field by providing a resilient and trustworthy solution for DFRL, opening up new possibilities for real-world deployments. ---