Intelligent Vehicular Routing and Optimal Resource Allocation Using Deep Reinforcement Learning for Urban VANETs

Abstract

The proliferation of connected and autonomous vehicles has intensified the demand for robust Vehicular Ad-hoc Networks (VANETs). A critical challenge in urban VANETs is the co-design of efficient routing protocols and optimal communication resource allocation under highly dynamic and resource-constrained conditions. Traditional routing algorithms often fail to adapt to rapid topological changes, while conventional resource allocation schemes are not cognizant of the specific data flow requirements of multi-hop routes. This paper proposes a novel integrated framework, Deep Reinforcement Learning-based Vehicular Routing with Optimal Resource Allocation (DRL-VRORL), to address this joint problem. We model the urban VANET environment as a Markov Decision Process (MDP) where intelligent agents on vehicles collaboratively learn routing and resource allocation policies. The core of DRL-VRORL is a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) architecture, enhanced with a centralized critic for coordinated learning and a distributed actor for scalable execution. The framework simultaneously optimizes for end-to-end packet delivery delay, packet delivery ratio (PDR), and network throughput. For resource allocation, we formulate a convex optimization problem for power and sub-channel allocation in Vehicle-to-Infrastructure (V2I) and Vehicle-to-Vehicle (V2V) links, which is solved efficiently using the Lagrange duality method, guided by the DRL agent's routing decisions. Extensive simulations conducted in SUMO and NS-3 demonstrate that DRL-VRORL significantly outperforms state-of-the-art protocols like AODV, DSR, and Q-learning-based routing. Specifically, DRL-VRORL achieves up to a 32% higher PDR, 45% lower average end-to-end delay, and 28% better aggregate network throughput while maintaining superior resource utilization efficiency.