Hierarchical Reinforcement Learning for Integrated Energy System Scheduling Based on Large Language Model Forecasting

Abstract

The uncertainties in source-side renewable power supply and user-side multi-energy demand pose significant challenges to coordinated scheduling in an electricity–heat–hydrogen integrated energy system (EHH-IES). A hierarchical scheduling approach for EHH-IES is introduced, with source–load forecasts serving as its basis. Traditional forecasting methods heavily rely on large amounts of training samples. To address the forecasting challenge in data-scarce scenarios, a frozen large language model assisted by variational mode decomposition is developed for joint source–load forecasting. During the scheduling process, conventional single-level reinforcement learning strategies are not sufficiently effective in dealing with the high-dimensional hybrid action space while satisfying the intricate operating constraints of the EHH-IES. Therefore, an imitation-learning-based hierarchical proximal policy optimization strategy is developed to decompose the scheduling task into system-level energy coordination and device-level action execution. The experimental evaluation shows that the proposed forecasting approach delivers improved forecasting accuracy in data-scarce scenarios, reducing the RMSE of photovoltaic power, wind power, electric load, heat load, and hydrogen load forecasting by 8.65%, 23.27%, 34.99%, 24.60%, and 39.38%, respectively, compared with the strongest baselines. The proposed scheduling strategy achieves the fastest convergence compared with the three benchmark methods while reducing the total operating cost by 21.3%, 7.1%, and 2.8%, respectively.