Risk-aware hierarchical reinforcement learning for investment and operations in cross-border energy supply chains under the belt and road initiative

Abstract

Cross-border energy supply chains under the Belt and Road Initiative (BRI) face unprecedented challenges, including geopolitical instability, macroeconomic volatility, and operational disruptions. Traditional optimization or flat reinforcement learning (RL) methods often struggle to address the scale, uncertainty, and multi-stakeholder complexity inherent in such environments. To overcome these limitations, this paper proposes a novel hierarchical reinforcement learning (HRL) framework that separates strategic investment planning from tactical operational control via a two-level architecture. The high-level agent makes long-horizon decisions on resource allocation and infrastructure deployment, while the low-level agent handles real-time logistics under dynamically imposed constraints. The framework explicitly incorporates multi-dimensional risk—political, economic, and operational—into both reward shaping and policy learning. Through extensive simulations on representative BRI corridors, the proposed HRL model demonstrates superior simulated performance over single-level RL, mixed-integer programming, and rule-based baselines across key metrics, including cumulative profit (+ 25%), service-level reliability, and robustness to stress scenarios. Ablation and continual learning experiments further confirm the contributions of hierarchical abstraction and risk-aware training. This work contributes a scalable, interpretable, and risk-resilient decision-making architecture for intelligent energy infrastructure planning in global contexts.