Jun 2026· arXiv.org· Vol abs/2606.29953· 0 citations· 54 references
Computer Science
Abstract
LLM-assisted evolutionary search (LES) has emerged as a promising paradigm for automated algorithm design. However, existing methods usually suffer from two inherent limitations when facing the automated design of real-world complex algorithms that usually consist of multiple components. The first limitation is that they either focus on modifying entire algorithms, making it difficult to reuse high-quality components, or concentrate on component refinement within a limited set of predefined multicomponent configurations. The second limitation is the insufficient explicit modeling and exploitation of algorithm semantics. These limitations severely degrade search efficiency and hinder effective exploration of complex design spaces. Therefore, this paper proposes STABLE (Semantics-Aware Bilevel Co-Evolution), an LES method purpose-built for automated multicomponent algorithm design that introduces structural algorithm formulation and semantics-driven evolution. In STABLE, complex algorithms are organized into hierarchical and modular architectures rooted in domain knowledge, aligning the search space with their intrinsic compositional traits. Based on this structured algorithm formulation, STABLE simultaneously optimizes high-level multicomponent configurations and low-level functional components, enabling coordinated cross-level updates while maintaining suitable granularities for design space exploration. At each level, STABLE establishes a multi-faceted semantic model to assist LLMs in capturing structural correlations, functional compatibilities, and inherent rationalities among algorithm components. This semantic model serves as the core guidance for evolutionary search, enabling principled algorithm generation and algorithm evaluation. Extensive experiments demonstrate that STABLE outperform both human-designed baselines and those from advanced LES methods.
This work proposes a general LES framework that incorporates feature-assisted niche construction within abstract search spaces, enabling the seamless integration of niche-based search strategies from evolutionary computation and introduces PartEvo (Partition to Evolve), an LES method that combines niche collaborative search and advanced prompting strategies to improve algorithm discovery efficiency.
Qinglong Hu, Qingfu Zhang· Neural Information Processin...· 9 citations· ⚡3
The rapid development of Large Language Models (LLMs) has opened new avenues for Automated Heuristic Design (AHD) for solving NP-hard combinatorial optimization problems (COPs). However, existing LLM-driven AHD methods are largely confined to rigid solver templates, relegating the search process to isolated module tuning. Transitioning to fully autonomous, system-level algorithm design is essential but fraught with low reliability of generated operators, extremely large search spaces, and ineffective credit assignment. To overcome these drawbacks, this paper proposes a Directed Graph-Guided Automated Algorithm Design framework, termed DGA$_2$D. It structures the open-ended program space as a directed graph, where each node represents a functional operator that can be instantiated using one of multiple candidate code implementations, while directed walks constitute complete algorithmic pipelines. A first-order path-dependent credit assignment mechanism is introduced to evaluate code variations strictly based on their topological context. Extensive experiments across 12 distinct COPs, ranging from complex scheduling to routing, demonstrate the consistent empirical advantages of DGA$_2$D. It reduces the average normalized gap by up to 10.96 percentage points compared to state-of-the-art LLM baselines.
Jiale Zhao, Zimu Chen, Sirui Mao et al.· 0 citations
Designing effective heuristics remains a labor-intensive task traditionally reserved for domain experts. While Large Language Model (LLM)-driven evolutionary search offers a path toward automated discovery, existing methods often suffer from slow convergence, primarily due to inefficient hyperparameter tuning. Delegating tuning to specialized optimizers improves performance, but it comes at the cost of code bloat and overfitting. To address these issues, we propose a pipeline that introduces a novel regularization architecture balancing performance and complexity. Specifically, we mitigate the side effects of automated tuning through two novel components: (i) symbolic pruning mutator, which combines LLM semantic guidance with Abstract Syntax Tree analysis to eliminate algorithmic redundancy; and (ii) a complexity-aware mutation gate that explicitly filters out mutations leading to excessive code growth. Our framework substantially accelerates convergence and improves generalization on standard benchmarks of the 3D Single Container Loading Problem. The discovered heuristics match state-of-the-art human designed algorithms and rediscover similar geometric principles used by experts, highlighting the framework's ability to autonomously extract meaningful domain knowledge.
Guorui Quan, Mingfei Sun, Manuel López-Ibáñez et al.· Annual Conference on Genetic...· 0 citations
Large Language Model (LLM)-based automated algorithm design typically evolves algorithms as complete, indivisible programs. While this whole-program perspective simplifies the search space, it fundamentally couples the useful local logic to its host program. Consequently, valuable code snippets vanish when the overall program is discarded, making it highly difficult to assess the contribution of individual algorithmic components.To address this, we propose Primitive-Aware Code Evolution (PACE), which decouples local logic from complete programs by representing it as persistent units called Executable Algorithmic Primitives (EAPs). To enable code-level transfer, PACE maintains a dynamic set of EAPs. Algorithm evolution is driven by primitive-aware operators that structurally guarantee the retention and cross-program transfer of these components. To evaluate them effectively, PACE leverages Thompson sampling based on parent-relative performance improvements, guiding primitive selection from the set without requiring extra evaluation datasets. Experiments on four tasks demonstrate that PACE effectively discovers competitive algorithms while structurally preserving valuable algorithmic components.
Zhuoliang Xie, Ruihao Zheng, Xiangzhao Xu et al.· 0 citations
Application Programming Interfaces (APIs) are central to modern software development, yet the growing scale and semantic complexity of APIs present significant challenges for developers in learning and using them effectively. API knowledge graphs (KGs) offer a structured solution by organizing API entities and their relations, but existing construction methods face a trade-off: schema-guided methods ensure precision but suffer from limited richness, while schema-free methods capture richer semantics but often lack reliability. To address this challenge, we propose Explore-Construct-Verify (ECV), a three-stage framework for API KG construction using large language models (LLMs). The framework first explores candidate schema in a bottom-up manner, then constructs the KG with schema-guided extraction, and finally applies human-AI collaborative verification based on association rule mining. This design preserves LLMs’ ability to discover domain-specific knowledge while enabling efficient post-hoc validation. We conduct extensive experiments to evaluate the effectiveness of our method. Compared to the state-of-the-art method EDC, ECV improves KG construction F1 score by 31.3%. Component analysis shows that the exploration module improves KG richness (recall) by 267.9%, the fully connected schema strategy increases richness by 50.0%, and the verification module improves KG reliability (precision) by 47.1%. ECV maintains an average F1 around 0.69 both across six programming languages and across five LLMs, demonstrating strong cross-language and cross-model generalizability. We further validate the practical utility of the constructed KG through an API recommendation task, where KG-enhanced LLMs outperform LLM-only baselines by 26.3% in accuracy.
Yanbang Sun, Qing Huang, Zhenchang Xing et al.· ACM Transactions on Software...· 0 citations
Large Language Models (LLMs) are opening new directions for automated heuristic design (AHD), allowing evolutionary methods to create and enhance heuristics for constrained optimization problems (COPs). However, most existing approaches face the challenge of the exploration-exploitation balance, where the evolution needs to escape convergence to homogeneous populations and discover as large a heuristic landscape as possible. To address this challenge, we introduce Quality-Diversity Evolution (QDEvo), a multi-objective framework that integrates Quality-Diversity optimization with LLM-based AHD. At its core, QDEvo employs a semantic survival selection mechanism that clusters algorithms by functional similarity, then applies local Pareto competition. Evaluation on well-known COPs benchmarks and real-world problems shows that our method consistently outperforms the state-of-the-art baseline in both Hypervolume and Inverted Generational Distance metrics. These results facilitate further exploration of the algorithmic design space, while ensuring competitive solution quality and efficiency.
Nam Do Khanh, Nhat Nguyen Tran Minh, Dat Pham Vu Tuan et al.· Proceedings of the Genetic a...· 1 citation