Neural Network Adaptive Topology Optimization (NNAUTO)

Abstract

This paper proposes a novel approach, Neural Network Adaptive Topology Optimization (NNAUTO), for designing and optimizing neural network architectures. The core idea is to dynamically adjust the topology of a neural network during training using a reinforcement learning (RL) agent, guided by a physics-informed energy-based optimization framework. Traditional neural network design often relies on fixed architectures or static designs, limiting adaptability to complex datasets and diverse tasks. NNAUTO addresses this limitation by enabling the network to evolve its structure in response to learning progress, optimizing for both accuracy and efficiency. The system employs an RL agent that learns optimal topology adjustments based on network performance metrics – such as accuracy and latency – and the corresponding changes in network topology, encompassing connection strengths and neuron counts. A key element is the integration of an energy-based optimization method, specifically a potential field approach, which constraints the learning process, ensuring that the generated topology modifications are physically plausible and aligned with biological neural network principles. This dynamic adaptation leads to significant improvements in network performance and efficiency compared to static or post-training architectural adjustments. The presented methodology offers a pathway towards more robust and adaptive neural networks, particularly for applications involving highly variable or complex input data.