DyCA treats instance clustering as a co-evolving component within the search process, reusing accumulated evaluation data as feature-free signals to progressively partition instances with similar algorithmic response patterns, thereby enabling finer-grained and more adaptive guidance for specialized algorithm design.
Abstract
Large Language Model-assisted Evolutionary Search (LES) has emerged as a powerful paradigm for automated algorithm design. However, existing LES methods primarily optimize for average performance, inherently directing search effort toward instances that contribute most to this metric while leaving others poorly served, resulting in weak tail robustness and limited real-world reliability. To address this limitation, we propose Dynamic Instance Clustering and Specialized Algorithm Design (DyCA), an LES framework with a feature-free, structure-aware mechanism for constructing reliable algorithm portfolios under heterogeneous instance distributions. DyCA treats instance clustering as a co-evolving component within the search process, reusing accumulated evaluation data as feature-free signals to progressively partition instances with similar algorithmic response patterns. The uncovered clusters decompose the mixed objective into a set of structure-aware sub-objectives, thereby enabling finer-grained and more adaptive guidance for specialized algorithm design. Experimental results across four algorithm design tasks with heterogeneous instances demonstrate that DyCA outperforms state-of-the-art LES baselines, improving tail robustness by an average of 15.2\% and overall performance by 7.1\% while maintaining competitive head performance.
This work proposes a general LES framework that incorporates feature-assisted niche construction within abstract search spaces, enabling the seamless integration of niche-based search strategies from evolutionary computation and introduces PartEvo (Partition to Evolve), an LES method that combines niche collaborative search and advanced prompting strategies to improve algorithm discovery efficiency.
Qinglong Hu, Qingfu Zhang· Neural Information Processin...· 9 citations· ⚡3
Automatic algorithm configuration is a well-established problem in optimization that becomes particularly challenging when dealing with heterogeneous sets of problem instances. In such scenarios, instance characteristics such as difficulty, structure, or dimensionality can induce substantial performance variability, leading to a trade-off between exploring diverse configurations for generalization and intensifying the search to improve performance. While specialized configurations for each instance type would be ideal, this is often unfeasible due to limited tuning budgets and the lack of reliable knowledge about instance similarity. To explore this issue, we propose an extension of the irace framework, h-irace, that incorporates concurrent racing processes over predefined instance subsets. The proposed approach is evaluated on two heterogeneous tuning scenarios using algorithms from the Ant Colony Optimization (ACO) and PSO-X families. The results show that the proposed method supports configuration specialization across heterogeneous instance subsets in both continuous and combinatorial optimization problems.
Diana Gil-Silvestre, Nicolás Rojas-Morales, T. Stützle et al.· Proceedings of the Genetic a...· 0 citations
Large Language Models (LLMs) are opening new directions for automated heuristic design (AHD), allowing evolutionary methods to create and enhance heuristics for constrained optimization problems (COPs). However, most existing approaches face the challenge of the exploration-exploitation balance, where the evolution needs to escape convergence to homogeneous populations and discover as large a heuristic landscape as possible. To address this challenge, we introduce Quality-Diversity Evolution (QDEvo), a multi-objective framework that integrates Quality-Diversity optimization with LLM-based AHD. At its core, QDEvo employs a semantic survival selection mechanism that clusters algorithms by functional similarity, then applies local Pareto competition. Evaluation on well-known COPs benchmarks and real-world problems shows that our method consistently outperforms the state-of-the-art baseline in both Hypervolume and Inverted Generational Distance metrics. These results facilitate further exploration of the algorithmic design space, while ensuring competitive solution quality and efficiency.
Nam Do Khanh, Nhat Nguyen Tran Minh, Dat Pham Vu Tuan et al.· Proceedings of the Genetic a...· 1 citation
The Automatic Construction of Portfolios via Large Language Models (LLM-ACP) suffers from poor generalization in practical few-shot scenarios when solving complex combinatorial optimization problems. Instance and algorithm co-evolution frameworks address this by expanding the training dataset with generated hard instances on which the current algorithm portfolio underperforms, thereby enhancing generalization. However, this paradigm faces two critical limitations: evaluating instance hardness relies on high-quality reference solutions, and single-mode generation patterns limit instance diversity. To overcome these limitations, we introduce the Potential-aware Instance and Algorithm Co-evolution (PIAC) framework. Our core contribution is twofold. First, we propose potential gain, a novel metric that eliminates the need for reference solutions. This metric estimates generalization gain by perturbing the generated algorithms and assessing their improvement potential on generated problem instances. Second, PIAC leverages LLMs to synthesize diverse instance mutators, exploring a broader region of the problem-instance space and thereby enhancing the portfolio's generalization capabilities. Given that perturbation spaces vary across different algorithms, we instantiate our framework on Greedy Constructive, Ant Colony Optimization, and Guided Local Search algorithmic backbones. Comprehensive evaluations on the Traveling Salesman Problem (TSP) and Capacitated Vehicle Routing Problem (CVRP) across six distinct data distributions demonstrate that PIAC consistently outperforms state-of-the-art LLM-ACP baselines, notably achieving a 19.76% relative improvement for TSP Greedy Constructive portfolios.
Shaofeng Zhang, Shengcai Liu, Zhiyuan Wang et al.· 0 citations
Large language model-based automated heuristic design (LLM-AHD) has shown strong potential in discovering effective heuristics for combinatorial optimization problems. However, existing methods primarily optimize a single heuristic, whereas practical optimization frameworks often rely on multiple interacting components. Directly extending single-heuristic methods is challenging because early component selection can overlook components with late potential, while independent evolution ignores inter-component dependencies. We propose MuEvo, an LLM-driven framework for evolving heuristic ensembles under ensemble-level feedback. MuEvo combines Dynamic Component Management, which uses short-budget probing and a reversible lifecycle to revise component priorities throughout the search, with LLM-Driven Co-Evolution, which coordinates component populations through Multi-Ensemble Evaluation, Cross-Component Information Sharing, Relation-Guided Pair Evolution, and Adaptive Budget Allocation. We evaluate MuEvo on selection hyper-heuristics and componentized ant colony optimization across four combinatorial optimization domains. Results show that MuEvo consistently improves human-designed frameworks and outperforms representative multi-component extensions of state-of-the-art LLM-AHD methods, demonstrating its effectiveness across both controller-mediated heuristic pools and functionally differentiated algorithmic components.
Haoze Lv, Ning Lu, Shengcai Liu et al.· 0 citations