Dynamic Bayesian Optimization with Reinforcement Learning

Abstract

Bayesian Optimization (BO) is a powerful technique for optimizing black-box functions, particularly those where gradient information is unavailable. However, its performance can degrade significantly when faced with noisy or non-stationary objective functions. This paper proposes a novel approach, Dynamic Bayesian Optimization with Reinforcement Learning (DBO-RL), to address these limitations. DBO-RL integrates Reinforcement Learning (RL) to enable the BO algorithm to dynamically adapt its exploration strategy. An RL agent is trained to learn an optimal exploration policy, balancing exploration and exploitation based on the observed behavior of the objective function. The core claim of this work is that by dynamically adapting the exploration strategy, DBO-RL achieves superior performance compared to traditional BO methods in scenarios with noisy or non-stationary objectives. The algorithm utilizes a Gaussian Process (GP) surrogate model to represent the objective function and the RL agent learns to select acquisition functions based on the GP predictions and uncertainties. The presented framework offers a robust and adaptive solution for optimization problems where the underlying function's characteristics are uncertain and prone to change.