RL-ABC: Reinforcement learning for accelerator beamline control

Abstract

Particle accelerator beamline optimization is a high-dimensional control problem traditionally requiring significant expert intervention. We present RLABC (Reinforcement Learning for Accelerator Beamline Control), an open-source Python framework that automatically transforms standard Elegant beamline configurations into reinforcement learning environments. RLABC integrates with the widely-used Elegant beam dynamics simulation code via SDDS-based interfaces, enabling researchers to apply modern RL algorithms to beamline optimization with minimal RL-specific development. The main contribution is a general methodology for formulating beamline tuning as a finite-horizon decision process: RLABC automatically preprocesses lattice files to insert diagnostic watch points before each tunable element, constructs a 57-dimensional observation from beam statistics, correlation information, and aperture constraints, and provides a configurable reward function for transmission optimization. The environment follows the Gymnasium interface and can therefore be coupled to continuous-control RL algorithms; the present study evaluates DDPG and implements an optional stage-learning curriculum. Validation on a test beamline derived from the VEPP-5 injection complex (37 control parameters across 11 quadrupoles and 4 dipoles) demonstrates that the framework successfully enables RL-based optimization, with a Deep Deterministic Policy Gradient agent achieving 70% particle transmission. The framework’s stage learning capability allows decomposition of complex optimization problems into manageable subproblems and provides an optional warm start for later training stages. The complete framework, including configuration files and example notebooks, is available as open-source software to facilitate adoption and further research.