Aug 2026· Proceedings of the Institution of mechanical engineers. Part D, journal of automobile engineering· 0 citations· 35 references
TL;DR
Simulation results demonstrate that, compared to the traditional hierarchical optimization framework, the proposed strategy achieves significant improvements in terms of mean absolute jerk, root-mean-square (RMS) value of acceleration, power demand, battery SOH degradation, battery temperature violation, and comprehensive operating cost.
Abstract
To address the synergistic optimization of car-following control and energy management, this study proposes a collaborative optimization framework based on deep reinforcement learning (DRL). At the car-following control level, a predictive cruise control (PCC) model is developed using the twin delayed deep deterministic policy gradient (TD3) algorithm, which incorporates safety and passenger comfort and power demand into the reward function. At the energy management level, a TD3-based energy management strategy (EMS) is formulated, incorporating constraints on battery state of health (SOH) degradation, temperature violations, state of charge (SOC) fluctuations, and comprehensive operating costs. Simulation results demonstrate that, compared to the traditional hierarchical optimization framework, the proposed strategy achieves significant improvements in terms of mean absolute jerk, root-mean-square (RMS) value of acceleration, power demand, battery SOH degradation, battery temperature violation, and comprehensive operating cost, with optimization rates of 32.7%, 49.55%, 5.39%, 28.30%, 91.76%, and 22.82%, respectively. Furthermore, generalization validation indicates that the proposed framework maintains strong robustness and adaptability under unknown driving conditions.
An integrated deep reinforcement learning (DRL) strategy in which a single Twin Delayed Deep Deterministic Policy Gradient (TD3) agent simultaneously adjusts the proportional and integral gains of the speed controller, the torque modulation coefficient (Ks), and the regenerative braking factor (βreg) is proposed.
Saber Hadj Abdallah, Fatma Ben Salem, Jaouhar Mouine et al.· Electronics· 0 citations
Microgrids play a critical role in enhancing the flexibility, reliability, and sustainability of modern power systems by integrating distributed energy resources, energy storage systems, and controllable loads. However, the inherent uncertainty of renewable generation and the stochastic nature of load demand pose significant challenges to optimal energy management. To address these issues, this paper proposes a deep reinforcement learning (DRL)-based optimal energy management framework for microgrids. The problem is formulated as a Markov decision process, where the system state captures renewable generation, load demand, and storage status, while the control actions determine power dispatch and energy storage operation. A deep reinforcement learning model is developed to learn optimal control policies through continuous interaction with the environment, enabling adaptive decision-making under dynamic and uncertain conditions. To improve learning efficiency and policy stability, state normalization and reward shaping strategies are incorporated. Furthermore, a constrained optimization mechanism is introduced to ensure operational safety and economic feasibility. Experimental results on benchmark microgrid scenarios demonstrate that the proposed method outperforms conventional rule-based strategies and model-based optimization approaches in terms of operational cost reduction, energy utilization efficiency, and robustness under uncertainty. The results indicate that the proposed DRL-based framework provides an effective and scalable solution for intelligent microgrid energy management.
Li Chen, Hong-Qiao Li, Zhen-Xing Chen et al.· European Conference on Elect...· 0 citations
To address the poor adaptability of traditional rule-based control, the operational instability of basic Q-learning algorithms, and the critical difficulty of deploying complex reinforcement learning models on resource-constrained on-board embedded platforms, this paper proposes a lightweight, simplified Q-learning energy management strategy for extended-range electric vehicles (REEVs), successfully implemented on an STM32 microcontroller. The algorithm achieves significant computational reduction by simplifying the traditional 5×5 state-action space into a highly condensed 2×2 grid. Furthermore, a power cooling mechanism is introduced, a multi-dimensional reward function is reconstructed to balance competing vehicle demands, and an ε-decay exploration strategy is designed. Software-in-the-loop (SIL) simulation verification demonstrates that the proposed strategy tightly controls the state-of-charge (SOC) standard deviation within 0.09. Additionally, high-frequency power fluctuations and range extender start-stop times are drastically reduced, and overall energy efficiency is improved by 50.3% compared with traditional strategies. The optimized algorithm occupies only 72.3% of RAM and 68.7% of Flash memory, fully satisfying strict on-board embedded system constraints and providing a highly feasible solution for intelligent REEV energy management.
Junyan Guo· European Conference on Elect...· 0 citations
HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67\% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6\% energy savings. The PPO policy satisfies formal safety certification with a margin of $2.003^\circ$C. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.
Oussama Ziadi, A. Rochd, S. I. Kaitouni et al.· 0 citations
Fuel cell electric vehicles (FCEVs) require energy management strategies that can balance hydrogen economy, battery utilization, component protection, and real-time control under varying driving conditions. This paper proposes an Adaptive-Modality Deep Deterministic Policy Gradient and Model Predictive Control hierarchical energy management strategy (AMDDPG–MPC HEMS). In the proposed architecture, the upper-level AMDDPG controller identifies driving-condition patterns and generates adaptive weights for hydrogen consumption, battery power, and state-of-charge regulation, while the lower-level MPC controller performs constrained power allocation between the fuel cell and battery. To improve adaptability, the AMDDPG algorithm incorporates an adaptive modality perception mechanism that extracts driving-condition features and a multi-scale reward mechanism that coordinates short-term energy-saving objectives with long-term component-protection requirements. A dedicated weight-scheduling and switching mechanism is also introduced to ensure smooth transitions between operating conditions. The proposed strategy is evaluated under the World Light Vehicle Test Cycle and Urban Dynamometer Driving Schedule and compared with rule-based, equivalent consumption minimization, and fixed-weight MPC strategies. The results show that the AMDDPG–MPC HEMS achieves the lowest equivalent hydrogen consumption, with reductions of 18.853% and 11.732% relative to the rule-based strategy under the two driving cycles, respectively. It also improves fuel-cell operating efficiency and maintains feasible battery SOC regulation. These results demonstrate the effectiveness and engineering potential of the proposed hierarchical energy management strategy.
Yan-Tao Si, Zhuo Wang, Changqun Sun et al.· Vehicles· 0 citations
To improve the ride comfort and attitude stability of the vehicle under complex driving conditions, this paper proposes a distributed model predictive control (DMPC) strategy with an adaptive weight-tuning mechanism based on the deep deterministic policy gradient (DDPG) algorithm for the active suspension system. The proposed method addresses the strong coupling among body vertical, pitch, and roll vibration-control objectives. It also reduces the reliance of conventional controllers on empirical parameter tuning and improves their adaptability to varying conditions. This study establishes a seven-degree-of-freedom full-vehicle active suspension model and decomposes it into a body subsystem and four wheel subsystems according to the coupling relationships. Then, a distributed predictive control framework is constructed. In this framework, local receding-horizon optimization and limited information exchange are used to achieve coordinated control. Furthermore, the DDPG algorithm learns the dynamic characteristics of the system online and adaptively adjusts the weighting parameters of the DMPC controller in real time. This enables dynamic allocation of control effort under varying operating conditions. The simulation results obtained from a high-fidelity CarSim co-simulation platform show that the proposed method effectively suppresses body vertical, pitch, and roll vibrations under different operating conditions. In addition, the proposed strategy reduces the average computation time compared with conventional MPC. Hardware-in-the-loop experiments further validate the effectiveness and real-time performance of the proposed controller.
Huichao Zhang, Jiayu Lu, Bo Li et al.· Journal of Vibration and Con...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
