Reinforcement Genetic Programming is proposed, a purely GP-based and non-RL framework that explicitly realizes a stage-wise retention– reintroduction loop through a dynamically maintained subexpression pool and pool-guided population initialization, suggesting that progressive subexpression reuse is an effective mechanism for SR search.
Combinatorial problems appear in numerous industrial applications. A common approach is to formulate these problems as declarative constraint models that can subsequently be compiled to and solved by a range of back-end solvers. Recent work shows that Large Language Models (LLMs) can produce correct models from natural language, but even a correct model can be expensive to solve because performance remains sensitive to modelling choices. In this work, we investigate whether LLMs can automate performance-oriented model reformulation. Inspired by Automatic Heuristic Design (AHD), we use an evolutionary framework in which an LLM proposes candidate reformulations that are verified and benchmarked against the user-defined baseline model. We compare AHD-adapted search strategies that control which prior attempts, instructions, and measured feedback enter each prompt. Existing retention strategies prioritize recency or performance, but do not explicitly diversify the context. To cover this gap, we introduce Profile-Diverse Retention (PDR), which applies Maximal Marginal Relevance (MMR) to instance-level runtime vectors to retain behaviourally diverse attempts. We systematically evaluate the strategies on eight CSPLib problems using validation-based final model selection. The results show that: (i) iterative reformulation can produce substantial held-out speedups; (ii) strategies that keep the retained context diverse outperform those that retain only recent or the fastest attempts; and (iii) validation-based selection improves the held-out speedup of every strategy.
Symbolic regression (SR) seeks to discover parsimonious mathematical laws from observational data, yet conventional approaches often struggle with the vast combinatorial search space of physically meaningful expressions. We present InsightSR, a framework that embeds Large Language Models (LLMs) as a guiding layer around the PySR genetic programming engine. Rather than relying on LLMs to generate expressions directly, InsightSR uses LLMs to progressively transform the search space itself through two complementary pathways: a Semantic Seed Pathway that proposes dimensionally consistent functional skeletons, and a Structural Feature Pathway that recommends nonlinear feature transformations. These transformations accumulate over iterations, broadening the input space and shifting the symbolic search from constructing deep expression trees over raw variables to assembling shallow trees over a rich, semantically informed feature set. A post-generation feedback loop evaluates candidates, categorizes features by their empirical utility, and refines the guidance for the next iteration, transforming the discovery process from open-ended generation into iterative, self-correcting refinement. Across three benchmarks, InsightSR achieves a 95% exact recovery rate on the Feynman benchmark and 80.18% accuracy on the LLM-SRBench LSR-Transform task, substantially outperforming state-of-the-art genetic programming and neural-symbolic methods while maintaining strong out-of-distribution generalization on real-world datasets.
Symbolic Regression (SR) is a central problem in evolutionary computation concerned with identifying symbolic equations from data. In many scientific and engineering settings, observed data are governed by physical laws drawn from known but implicitly defined families of equations, where the exact symbolic form is unknown and difficult to enumerate a priori. In such settings, exact structural identification is more critical than arbitrary function approximation. We present an evolutionary SR approach for scientific equation identification in which large language models (LLMs) are integrated directly into the evolutionary process as guided variation operators. The method maintains a population of candidate symbolic expressions and evolves them over successive generations using fitness-based selection, structural diversity preservation, and stateful evolutionary memory. LLM guidance proposes structurally informed variations that exploit qualitative prior knowledge while remaining embedded within an evolutionary search framework. We evaluate the approach on the SRSD-Feynman benchmark, demonstrating robust rediscovery of scientific equations across easy and medium difficulty subsets, with competitive performance on harder instances. Results show that incorporating language-model guidance into evolutionary SR substantially improves search efficiency while preserving interpretability and the evolutionary character of the algorithm.
Jun Zhao, Kei Sen Fong, M. Motani· Proceedings of the Genetic a...· 0 citations
Analysis shows that many generated operators use semantics to guide selection, suggesting that LLMs can produce non-trivial search heuristics from the task description alone, and the relationship between public LLM leaderboard rankings and GP performance is examined.
This work introduces an Evolutionary Language Model that searches over natural-language policy descriptions and compiles typed programs for execution and shows that language can serve as a steerable, execution-grounded search representation over executable program space.
Matthew Siper, A. Khalifa, Julian Togelius· 0 citations
Symbolic Regression (SR) aims to discover explicit mathematical expressions that describe the relationship between input variables and a target output, offering an interpretable alternative to black‐box machine learning models. Genetic Programming (GP) has been widely adopted for this purpose; however, traditional GP‐based approaches often suffer from high computational cost, limited scalability, and excessive model complexity. To address these limitations, this work presents M5GP 2.0, an extended version of the Multidimensional Multivariate Genetic Programming framework that builds upon the constructive feature induction paradigm introduced in M5GP (Parallel Multidimensional Genetic Programming with Multidimensional Populations for Symbolic Regression). M5GP 2.0 significantly expands the original framework by extending the evolutionary search space through more expressive and higher‐arity operators and introducing advanced GPU‐based optimisations for efficient large‐scale execution. The proposed method evolves symbolic feature transformations that are subsequently combined using linear models, enabling the generation of compact solutions with interpretability potential while maintaining competitive predictive performance. The experimental evaluation is conducted using a standardised and widely accepted benchmark, namely SRBench, enabling rigorous and reproducible comparisons against state‐of‐the‐art methods considering metrics such as predictive R2$$ {R}^2 $$ , root mean squared error (RMSE), model size, and training time. Overall, the results indicate that M5GP 2.0 constitutes a robust and scalable symbolic regression framework that achieves a favourable balance between predictive performance, model compactness, and computational efficiency, while retaining the potential for symbolic traceability and post hoc interpretability.
L. Florido, Leonardo Trujillo, Javier Carmona Troyo et al.· Expert systems· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
