Can Cooperative Routing Mitigate System-Level Inefficiency in Road Networks?

Abstract

Traffic congestion in urban road networks represents a fundamental coordination failure, where the aggregate behavior of self-interested agents results in a high “Price of Anarchy” that compromises system-wide efficiency. In decentralized settings, agents typically maximize local utility without accounting for the congestion externalities they impose on others, creating a non-stationary decision-making environment that is notoriously difficult to optimize. While classical traffic assignment and game-theoretic approaches quantify and analyze the efficiency gap, they often rely on static assumptions, centralized computation, or externally imposed incentives, which limits their applicability in the stochastic nature of real-world networks. This motivates the need for learning-based decentralized coordination mechanisms capable of navigating the dynamic complexities of modern urban traffic. In this work, we investigate how cooperative Multi-Agent Reinforcement Learning (MARL) can mitigate reduces system-level inefficiencies in a dynamic traffic assignment setting. Using a controlled traffic network with competing routes, incorporating traffic signal control and stochastic driver behavior, we model vehicles as decentralized agents and systematically evaluate reward and state design strategies under strictly controlled conditions. Our results show that purely selfish objectives fail to produce stable or efficient routing policies, whereas global cooperative objectives consistently yield significant improvements, achieving up to 24.3% reduction in Total Trip Time (TTT) compared to a state-of-the-art selfish route choice baseline. Beyond performance improvement, we identify a key design principle for cooperative MARL: system-level coordination can be achieved without complex reward shaping. Specifically, embedding Marginal Social Cost (MSC) into the agents’ observation space – rather than using it as an additional reward term – yields more stable learning and improved coordination compared to standard reward-only formulations. This approach improves learning stability, reduces variance, and delivers a reproducible 9.7% improvement compared to the baseline, demonstrating that exposing agents to system-level information can be a competitive alternative to reward-engineering-based approaches in this dynamic setting.