Large language model-based automated heuristic design (LLM-AHD) has shown strong potential in discovering effective heuristics for combinatorial optimization problems. However, existing methods primarily optimize a single heuristic, whereas practical optimization frameworks often rely on multiple interacting components. Directly extending single-heuristic methods is challenging because early component selection can overlook components with late potential, while independent evolution ignores inter-component dependencies. We propose MuEvo, an LLM-driven framework for evolving heuristic ensembles under ensemble-level feedback. MuEvo combines Dynamic Component Management, which uses short-budget probing and a reversible lifecycle to revise component priorities throughout the search, with LLM-Driven Co-Evolution, which coordinates component populations through Multi-Ensemble Evaluation, Cross-Component Information Sharing, Relation-Guided Pair Evolution, and Adaptive Budget Allocation. We evaluate MuEvo on selection hyper-heuristics and componentized ant colony optimization across four combinatorial optimization domains. Results show that MuEvo consistently improves human-designed frameworks and outperforms representative multi-component extensions of state-of-the-art LLM-AHD methods, demonstrating its effectiveness across both controller-mediated heuristic pools and functionally differentiated algorithmic components.
Large Language Models (LLMs) are opening new directions for automated heuristic design (AHD), allowing evolutionary methods to create and enhance heuristics for constrained optimization problems (COPs). However, most existing approaches face the challenge of the exploration-exploitation balance, where the evolution needs to escape convergence to homogeneous populations and discover as large a heuristic landscape as possible. To address this challenge, we introduce Quality-Diversity Evolution (QDEvo), a multi-objective framework that integrates Quality-Diversity optimization with LLM-based AHD. At its core, QDEvo employs a semantic survival selection mechanism that clusters algorithms by functional similarity, then applies local Pareto competition. Evaluation on well-known COPs benchmarks and real-world problems shows that our method consistently outperforms the state-of-the-art baseline in both Hypervolume and Inverted Generational Distance metrics. These results facilitate further exploration of the algorithmic design space, while ensuring competitive solution quality and efficiency.
Nam Do Khanh, Nhat Nguyen Tran Minh, Dat Pham Vu Tuan et al.· Proceedings of the Genetic a...· 1 citation
This work proposes a general LES framework that incorporates feature-assisted niche construction within abstract search spaces, enabling the seamless integration of niche-based search strategies from evolutionary computation and introduces PartEvo (Partition to Evolve), an LES method that combines niche collaborative search and advanced prompting strategies to improve algorithm discovery efficiency.
Qinglong Hu, Qingfu Zhang· Neural Information Processin...· 9 citations· ⚡3
Large Language Models (LLMs) are increasingly used as generative operators in evolutionary algorithms, yet the role of selection in LLM-assisted systems remains underexplored, particularly when optimizing structured natural language artifacts without ground-truth fitness signals. We introduce EvoHIIT, an LLM-assisted evolutionary framework for the design of High-Intensity Interval Training (HIIT) programs, and study preference-based selection mechanisms. Within an evolutionary loop, EvoHIIT compares four evaluation regimes: (i) scalar fitness based on LLM-predicted physiological and structural proxies, (ii) LLM-based pairwise qualitative selection, (iii) human preference selection, and (iv) hybrid strategies combining automated optimization with periodic human correction. Using schema-constrained HIIT programs as a controlled domain, we analyze convergence dynamics, mutation behavior and alignment with user and expert judgments. Our results show that the choice of selection mechanism substantially shapes search trajectories in semantic solution spaces. While proxy-based scalar fitness enables stable optimization, preference-based selection mitigates specification bias and improves perceived quality. Our findings position selection as a central design dimension in LLM-assisted evolutionary optimization and provide empirical insight into human-aligned evolution of natural language solutions.
Johana Chen, Niki van Stein, Robert Cabri et al.· Proceedings of the Genetic a...· 0 citations
Designing effective multi-objective Bayesian optimization (MOBO) algorithms requires balancing many interdependent design choices whose optimal configuration is problem-dependent and typically demands deep expertise. We extend the LLaMEA framework to MOBO, using large language models as mutation and crossover operators within evolutionary strategies to generate complete algorithm implementations, with SMAC hyperparameter optimization integrated into the evolutionary loop. Across nine evolutionary runs we generated approximately 900 algorithms and benchmarked them on twelve synthetic problems (ZDT, DTLZ, WFG) and three real-world engineering problems (RE), using a BoFire qParEGO implementation as a state-of-the-art Bayesian-optimization baseline. On the synthetic suite the strongest generated algorithm attains the highest mean normalized hypervolume (0.971, vs. 0.869 for qParEGO) while requiring roughly 60x less wall-clock time; a Friedman test with post-hoc analysis places the two in a single top-performing group, and per-problem tests find the generated algorithm significantly better than qParEGO on 7 of the 12 problems and never worse, matching state-of-the-art accuracy at an order-of-magnitude lower cost. On the three unseen real-world engineering problems a generated algorithm attains the best mean normalized hypervolume (0.985, vs. 0.971 for qParEGO)--significantly better than qParEGO on two of the three problems--at roughly 3.4x lower wall-clock cost, confirming that the gains transfer beyond the synthetic regime. LLM-driven evolutionary search can thus discover algorithm designs that achieve Pareto-efficient trade-offs difficult to reach through manual design.
G. Laskaris, R. Brasher, N. V. Stein et al.· 0 citations
Experimental results demonstrate that the proposed framework consistently outperforms classical ALNS baselines and several competitive metaheuristic methods in terms of solution quality and convergence speed.
T. M. Nguyen· Annual Conference on Genetic...· 0 citations
Configuring genetic algorithms (GAs) for new optimization problems typically requires substantial expertise in representation design, decoder construction, and constraint handling. We present a framework that combines large language models (LLMs) for problem understanding with a structured Biased Random-Key Genetic Algorithm (BRKGA) configurator for algorithm realization, allowing users to describe optimization problems in natural language and receive executable GPU-accelerated GA implementations. We compare two workflows: LLM-Only, where the LLM generates a complete standalone GA from the problem description; and LLM+Config, where the LLM is restricted to problem parsing and decoder synthesis against a fixed brkga3 interface that provides the evolutionary architecture, Adaptive Large Neighborhood Search (ALNS) warm-start, GPU-parallel segmented sort, and bottleneck-focused local search. We evaluate on three problem families (the Traveling Salesperson Problem (TSP), the Traveling Salesperson Problem with Job-times (TSPJ), and the Vehicle Routing Problem with Resource Pick-up and Delivery (VRP-RPD)) on instances up to 999 nodes. LLM+Config outperforms or matches LLM-Only on every instance where both workflows produced a result, and additionally solves four TSP instances on which LLM-Only failed (Wilcoxon signed-rank with Holm correction across the three problem families, all adjusted p ≤ 0.012); on TSPJ it matches CPLEX optimal on 3 of 10 benchmarks and establishes first reported results for 6 large instances (n = 202–1,000); on VRP-RPD it wins all 13 instances against LLM-Only with 51–329 % advantage on large instances. GPU-parallel BRKGA evolution runs 25–35× faster per generation than a single-threaded CPU baseline. We characterise LLM stochasticity through two complementary studies: 15 sessions across 3 problems with one model (80% success rate), and 100 sessions across 10 LLMs from 3 providers on TSPJ/eil101 (Friedman and Wilcoxon-Holm tests on within-tier and within-provider blocks), finding that capability tier dominates over provider at the large tier and that small-tier non-Anthropic models contribute zero usable sessions.
Harishjitu Seesandrn, M. Sodhi, Resit Sendag· Proceedings of the Genetic a...· 0 citations