Dynamic Bayesian Optimization with Reinforcement Learning for Hyperparameter Tuning

Abstract

Hyperparameter tuning is a critical, yet computationally expensive, component of modern machine learning workflows. Traditional Bayesian Optimization (BO) methods, while effective in many scenarios, often struggle when dealing with high-dimensional hyperparameter spaces, leading to slow convergence and suboptimal performance. This paper proposes a novel approach combining Bayesian Optimization with Reinforcement Learning (RL) to address this challenge. The core idea is to leverage an RL agent to learn an optimal exploration strategy for the BO process, dynamically adapting to the specific characteristics of the hyperparameter space. This allows the algorithm to navigate complex landscapes more efficiently and effectively. We formulate the problem as a Markov Decision Process (MDP), where the BO surrogate model acts as the environment, and the RL agent learns to select the next acquisition function based on its observed rewards. The proposed method demonstrates improved performance in high-dimensional hyperparameter tuning tasks compared to standard BO approaches.