Jul 2026· Signal Processing and Communications Applications Conference· pp. 1-4· 0 citations· 22 references
Abstract
Deploying reinforcement learning (RL) agents for heating, ventilation, and air-conditioning (HVAC) control across climatically diverse buildings is challenging, as policies trained in one climate often fail to generalize to others. We investigate transfer and meta-reinforcement learning for building climate control using Sinergym with a custom EnergyPlus model of a real room, comparing Double Deep Q-Network (DDQN) and Model-Agnostic Meta Learning DDQN (MAML-DDQN) across three transfer scenarios on a target very hot climate. Fine-tuned DDQN reduces temperature violations by a factor of 35 over the from-scratch baseline and improves reward by 86% over zero-shot transfer, while MAML-DDQN converges in less than half the fine-tuning episodes and exhibits substantially lower variance across seeds. Results highlight a key trade-off: MAML-DDQN is preferable at adaptation-focused deployments, while fine-tuned DDQN is superior when maximum performance is the objective.
This study proposes an explainable, data-driven framework integrating active-reward proximal policy optimization (AR-PPO), which successfully distills black-box AI strategies into verifiable, physics-informed standard operating procedures (SOPs), providing a highly transparent and robust solution for autonomous windshear escape and future competency-based flight training.
Reinforcement learning (RL) has achieved strong results in control, yet learned policies remain brittle to changes in dynamics, action spaces, observation spaces, or goals, a critical limitation for real-world deployment. Existing benchmarks offer limited diversity and complexity, making it difficult to rigorously study transfer, multi-task learning, and meta-learning in RL. We introduce Building2Building (B2B), a large-scale suite of realistic Heating, Ventilation, and Air Conditioning (HVAC) control environments built on EnergyPlus, a state-of-the-art building simulator. B2B is fully compatible with the Gymnasium interface and features a parametric building generator, enabling the systematic generation of diverse building configurations with heterogeneous observation and action spaces. Based on this suite, we define benchmark tasks targeting key open challenges in RL, including goal adaptation, dynamics adaptation, action-space shifts, and cross-domain transfer. By providing a large-scale, diverse, and physically grounded testbed with standardized evaluation protocols, B2B enables systematic investigation of generalization and transfer in continuous control. Beyond advancing research on generalization in RL, this new benchmark also carries significant societal implications by enabling improved HVAC control at scale, one of the most energy-intensive systems in buildings.
Vincent Taboga, Justine Veilleux, Doseok Jang et al.· 0 citations
This study adapts an open-weight reasoning model through reinforcement learning with verifiable rewards (RLVR), which converts exact offline dynamic-programming action values into dense rewards for every candidate action to adapt open-weight reasoning models to building storage scheduling.
Takumi Shioda, Kohei Terashima, T. Nagai· arXiv.org· 1 citation
HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67\% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6\% energy savings. The PPO policy satisfies formal safety certification with a margin of $2.003^\circ$C. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.
Oussama Ziadi, A. Rochd, S. I. Kaitouni et al.· 0 citations
Contrastive Reinforced Policy Optimization (CRPO) is introduced, which reformulates agentic OPSD from a contrastive learning perspective, and conducts group-wise contrast to preserve reliable, fine-grained optimization signals.
Xingjian Wu, Junlin Liu, Xing-Chen Liu et al.· arXiv.org· 1 citation
This paper proposes a contextual quality-diversity evolutionary reinforcement-learning controller, CQD-ERL, for the supervisory control of a tropical, water-cooled chiller plant and its associated air side. Rather than converging to a single scalarised policy, the controller maintains a product archive of specialised policies indexed jointly by a data- driven operating context, a cluster of daily weather and load regime, and a context-invariant behaviour descriptor, filled by a gradient-free evolutionary operator and a soft-actor-critic policy-gradient operator that share one replay buffer. Every action is filtered through a deterministic safety shield before execution. The controller is trained on a two-tier reduced-order environment representing the latent load, cooling-tower approach and humidity constraints of a Singapore commercial building, and is evaluated over a full annual backtest against an ASHRAE Guideline 36 baseline.
Tran Vu· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
