Iterated agent for symbolic regression

Abstract

Symbolic regression (SR), the automated discovery of mathematical expressions from data, is a cornerstone of scientific inquiry. However, it is often hindered by the combinatorial explosion of the search space and a tendency to overfit. Popular methods, rooted in genetic programming, explore this space syntactically, often yielding overly complex, uninterpretable models. This paper introduces IdeaSearchFitter, a framework that employs large language models as semantic operators within an evolutionary search. By generating candidate expressions guided by natural-language rationales, our method guides the search towards models that are not only accurate but also conceptually coherent and interpretable. We demonstrate IdeaSearchFitter’s efficacy across diverse challenges: it achieves competitive, noise-robust performance on the Feynman SR Database, outperforming several strong baselines; identifies candidate mechanistically aligned models with good accuracy-complexity trade-offs on real-world data; and derives compact, physically motivated representations of parton distribution function grids in a high-energy physics application. IdeaSearchFitter is a specialized module within our broader iterated agent framework, IdeaSearch, which is publicly available at: https://www.ideasearch.cn/.