Transformer Guided Multi-Fitness Genetic Programming for Symbolic Regression

Abstract

Symbolic regression aims to discover mathematical expressions that are both accurate and interpretable from data, holding significant value for scientific discovery and engineering modeling. Traditional methods primarily rely on Genetic Programming (GP), which, however, often suffers from low search efficiency in high-dimensional settings and when dealing with complex symbols. Recently, Transformer-based deep learning methods have offered a new approach with powerful sequence generation capabilities and fast inference speed, yet their expression accuracy often struggles to match that of GP. To integrate the strengths of both paradigms, this paper proposes a Multi-Fitness Transformer-guided Genetic Programming (MFTGP) framework. The core idea of this method is to train a Transformer model to obtain prior sign vectors, then simultaneously consider both numerical error and sign similarity as fitness criteria in GP. Experimental results on 20 functions demonstrate that MFTGP surpasses the compared algorithms in both search efficiency and solution accuracy, demonstrating a clear advantage.