Hybrid Deep Learning and First-Principles Model for Predictive Control of Solar Direct Steam Generation

Physics-Informed Deep Learning and Rule-Based adjustment PIDL-RB for online Building Energy Management System: A Comparative Study

Using Data-Driven Simulation Models for Deep Reinforcement Learning Based HVAC Control

This manuscript summarizes ongoing doctoral research on control of building heating, ventilation, and air conditioning (HVAC) systems through deep reinforcement learning trained on data-driven simulators. The thesis investigates whether multivariate time-series forecasting models can act as reliable surrogates of physics-based building emulators for controller development. The work is motivated by the high modeling effort required by conventional model predictive control and by the practical impossibility of training reinforcement learning agents directly on real buildings. The proposed methodology combines synthetic data generation from established simulation frameworks, fine-tuning of time-series foundation models, and cross-platform evaluation against high-fidelity building emulators. Current progress includes a published dataset-generation study and a submitted first paper centered on zero-shot forecasting of indoor temperature and HVAC energy consumption with Tiny Time Mixers across previously unseen buildings and seasons. A later stage of the thesis will study how reinforcement learning agents can use the learned surrogates for planning and HVAC control, and whether policies trained through those models transfer back to trusted simulation environments.

Comparative Study of Deep Q-Network-Based Reinforcement Learning and Model Predictive Control for Zone Comfort and Energy Optimization in VAV-Controlled HVAC

HVAC systems account for a major part of building energy consumption, making advanced control strategies essential. This paper compares Deep Q-Network (DQN)-based reinforcement learning and model predictive control (MPC) for thermal comfort and energy optimization in a VAV-controlled HVAC system. Both approaches are evaluated on a realistic smart building platform under dynamic conditions, including real outdoor temperature data and varying occupancy profiles. Results show that MPC achieves smoother control actions and better energy efficiency compared to DQN, whose discretized action space introduces oscillatory behavior. However, DQN demonstrates greater robustness to uncertainty without requiring an explicit system model. These findings suggest that hybrid strategies combining both approaches could further enhance real-world HVAC control performance.

Twin-delayed deep deterministic policy gradient for enhanced power optimization in solar PV-integrated DFIG wind energy systems.

The electrical power systems are facing rising challenges of stability and control with increasing share of intermittent renewable energy power sources. This work presents application of Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithm in single unified controller for multi-objective control of DFIG-Solar PV system connected to power grid. The commonly used Proportional-Integral (PI) controllers are not suitable to address nonlinearities of single controller based hybrid DFIG and solar PV systems. At times, the latest reinforcement learning-based controllers like DDPG can be erratic and aggressive due to overestimation of the actor's control action. These aggressive actions, which cause overshoot and oscillation, can be overcome by adopting the TD3 algorithm. The TD3 algorithm provides improved learning capabilities and performance by mitigating overestimation by using dual critic networks. A single TD3-based controller is implemented to simultaneously control the Rotor Side Converter (RSC), Grid Side Converter (GSC) and solar PV system integrated at the DC link. OPAL-RT real-time hardware-in-the-loop (HIL) simulation results demonstrate that the TD3 controller achieves a 10.3% reduction in power overshoot, 8% improvement in DC link voltage regulation, 15.3% faster response time, and 16.9% faster settling time compared to conventional PI control, and also outperforms the DDPG-based controller across all metrics.

Intelligent Control of Parabolic Trough Collectors via Deep Reinforcement Learning

: The effective control of parabolic trough collectors (PTCs) remains a significant challenge due to the inherent non-linearities of the system and the continuous impact of environmental disturbances. Although PTCs are a key technology for industrial process heat and large-scale electricity generation, classical control strategies often struggle to maintain optimal performance under fluctuating conditions. To address these limitations, this paper presents a novel reinforcement learning (RL)-based controller, designed specifically for solar thermal systems. The proposed RL agent is designed to learn directly from operational data, enabling it to adapt its control policy in real time to mitigate external disturbances. Experimental results demonstrate that the RL controller achieves a fast and well-damped closed-loop response, significantly outperforming traditional control benchmarks. Specifically, the RL controller is compared with a Proportional-Integral controller combined with a feedforward controller, and with a Model-based Predictive Controller. In all simulation-based comparisons, the RL controller outperforms the aforementioned controllers in terms of setpoint tracking or disturbance rejection. These results highlight the potential of machine learning to improve the operational reliability and efficiency of complex renewable energy systems.

Ultra-Short-Term Photovoltaic Power Forecasting Based on a Hybrid Decomposition Linear Long Short-Term Memory Model

The rapid expansion of photovoltaic (PV) systems poses significant challenges to grid stability. Hybrid Energy Systems (HES) are intended to alleviate this volatility, yet their coordinated dispatch often remains suboptimal due to communication delays and ramp-rate constraints. Accurate ultra-short-term PV power forecasting is therefore essential, as it enables preemptive control and timely dispatch adjustments that unlock the full potential of HES. In this study, we propose a novel AI hybrid forecasting framework that integrates a rule-based model with a Decomposition Linear (DLinear) Long Short-Term Memory (LSTM) deep learning core, representing, to the best of our knowledge, a novel integration of a decomposition-based linear model (DLinear) with LSTM networks for ultra-short-term PV power forecasting. The DLinear component decomposes the time series into trend and remainder sequences, which are then independently modeled by separate LSTM networks to capture distinct dynamics. Using data from a 300 kWp PV power station, the framework achieves an average daily prediction accuracy exceeding 93% for both 5-min and 15-min horizons. The model reliably tracks power variations under sunny and rainy conditions, while under volatile cloudy weather its accuracy decreases but still captures essential fluctuation patterns. These results demonstrate the potential of the proposed framework for improving the dispatch and operational reliability of hybrid energy systems. However, further validation across additional seasons and sites is needed to establish broader generalizability.