Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems
It is shown that under GRPO-style optimization, a global normalization baseline may deviate from diverse agents'reward distributions, which ultimately leads to gradient-norm instability, which theoretically pinpoints a key reason for training instability when extending group-based RL to multi-agent LLM systems.