Meta-Learning with Bayesian Optimization for Neural Network Architecture Search

Abstract

This paper presents a novel approach to neural network architecture search (NAS) leveraging meta-learning and Bayesian optimization. Traditional NAS methods often suffer from high computational costs associated with exhaustive or reinforcement learning-based exploration of the architecture search space. Our method, Bayesian Optimization with a Gaussian Process surrogate model, offers a significantly more efficient alternative. We learn from previous architecture search trials, using this knowledge to guide the selection of promising architectures in subsequent searches. This meta-learning framework allows us to rapidly converge on high-performing architectures, reducing the overall search time while maintaining competitive accuracy. The core of our approach lies in the use of a Gaussian Process to model the performance of different neural network architectures, and then employing an acquisition function to intelligently guide the exploration of the architecture space. This paper details the formulation of the problem, the implementation of the Bayesian optimization algorithm, and demonstrates its effectiveness through theoretical analysis and a discussion of the key components.