Intelligent planning and scheduling system for supply chain logistics routes based on reinforcement learning

Abstract

The optimization of logistics routing and scheduling in supply chains is a critical challenge, particularly in dynamic environments characterized by fluctuating demand, traffic conditions, and stringent time-capacity constraints. Traditional optimization methods and heuristic-based approaches often struggle to adapt and maintain stability in non-stationary settings. Reinforcement learning models, while promising, face limitations in scalability, sensitivity to reward variance, and generalization under dynamic conditions. This research proposes the Dynamic Actor-Critic Proximal Policy Optimization (DACA-CPPO) framework to address these challenges. The Dynamic Actor-Critic algorithm (DACA) component improves policy and value adaptation by dynamically updating the actor and critic networks, while the Clipped Proximal Policy Optimization (CPPO) stabilizes policy updates through clipping, ensuring that the policy updates remain within a stable range and preventing large, destabilizing changes. The framework is evaluated using a dataset of 32,065 records with 26 features, split into 80% for training and 20% for testing. Data preprocessing techniques, including Z-score normalization and feasibility filtering, ensure consistent scaling and valid state transitions across training and testing sets. Node2Vec-based graph embeddings are employed to extract spatial and topological features of logistics networks, while statistical and temporal variables such as demand, capacity utilization, and scheduling urgency are incorporated into the state representation. The results show that the DACA-CPPO framework achieved 95% accuracy, reduced delivery costs, lowered delay rates, and improved route efficiency compared to baseline reinforcement learning strategies. Implemented in Python, the framework offers a stable, scalable, and adaptive solution for optimizing logistics planning in dynamic, real-time environments. Not applicable.