Jun 2026· arXiv.org· Vol abs/2606.26578· 0 citations· 51 references
Computer Science
Abstract
Automating optimization modeling from natural language with large language models (LLMs) faces two key challenges. First, training corpora lack structural diversity. Second, data generation pipelines remain static and decoupled from model learning. To address these challenges, we propose EvoOptiGraph, a novel framework where data and model co-evolve, driven by model weaknesses. EvoOptiGraph represents each mixed-integer linear program (MILP) as an attributed bipartite graph and applies validity-preserving evolutionary operators to generate structurally diverse instances. The evolved graphs are converted into solver code and natural language via deterministic compilation and verified back-translation. Training proceeds in two stages: supervised fine-tuning (SFT) on an initial dataset, followed by reinforcement learning with verifiable rewards (RLVR), where graph-derived weakness signals guide the generation of new instances targeting the model's failures. This forms a closed loop that continuously updates the training distribution. Empirical results on six public datasets show that EvoOptiGraph significantly outperforms larger generalist models, agentic methods, and specialized baselines in accuracy, executability, and generalization. These results demonstrate that targeted data-model coevolution is an effective strategy for improving LLMs on optimization modeling tasks.
Evolutionary program search guided by Large Language Models (LLMs) has emerged as a powerful paradigm for automated scientific discovery. However, current approaches are fundamentally constrained by three bottlenecks: structurally blind parent selection, sparse whole-program evaluation rewards, and static mutation operators that fail to adapt during search. We present GAE (Graph-Augmented Evolution), a framework that resolves these limitations through a tightly coupled, three-pillar architecture. First, a relational graph neural network (GNN) parses programs into typed computation graphs, producing structure-aware embeddings. Second, an RL-optimized meta-controller leverages these embeddings to replace blind evolutionary sampling with a directed policy, dynamically selecting optimal parents and mutation directions based on reward history. Third, an online GRPO fine-tuning loop continuously updates the LLM mutation operator at test-time using group-normalized evaluation rewards, directly aligning the model's generation distribution with high-fitness structural edits. We evaluate GAE on a challenging scientific discovery task: symbolic regression for complex nonlinear oscillator systems. By transforming stochastic search into a directed, self-improving trajectory, GAE efficiently discovers closed-form physical equations, consistently matching or outperforming static LLM-driven baselines and achieving state-of-the-art out-of-distribution performance.
Large language models can improve with reinforcement learning for search agents, yet existing self play agents repeatedly generate tasks while discarding the knowledge gained during successful searches. We introduce CoEvoKG, a framework that turns a knowledge graph into both a source of verifiable training tasks and a persistent evidence memory for agent evolution. CoEvoKG jointly trains a task generator and a search agent: the generator creates multihop questions from entity chains sampled from the knowledge graph, while the agent learns from rewards for answer correctness and search trajectories whose entity paths are supported by graph evidence. When a search succeeds, CoEvoKG verifies and deduplicates the retrieved evidence, then writes it back to the corresponding graph nodes and edges. Future rounds reuse this enriched graph for task generation and reward computation, closing the loop between model self evolution and knowledge accumulation. Experiments on six QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, and Bamboogle) with three backbone models show that CoEvoKG improves macro average accuracy over the corresponding base models by +11.2, +10.1, and +11.6 points on Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, and Llama-3.1-8B-Instruct, respectively. Under matched training budgets, CoEvoKG further improves over competitive self play baselines and RL baselines for search agents by +2.6 to +3.7 macro average points across the three backbones. Code is available at https://github.com/lazzy1225/CoEvoKG.
Zhaoyang Li, Zenghuang Fu, Qiuyuan Ai et al.· 0 citations
Graph Structure Learning (GSL) is fundamental to effective graph-based modeling, particularly for life omics analysis. Traditional heuristics-based GSL methods impose overly rigid inductive biases, while optimization centric GSL suffers from non-convexity and inherent interpretability challenges. Evolutionary algorithms (EAs) have emerged as a promising alternative for their robust global exploration and ability to navigate discrete search spaces without gradient information. However, such stochastic mutations essentially bottleneck evolutionary efficiency. Furthermore, both optimization-based and existing evolutionary paradigms lack semantic guidance, resulting in inexplainable graph structures. Specifically, the topological modifications tend to be arbitrary, failing to reveal the underlying algorithmic logic required for biological insights. To address these problems, we propose CEAgent-GSL, a Code-level Evolutionary LLM Agent framework. CEAgent-GSL treats graph generation as an evolving population of executable scripts, leveraging an LLM for semantic mutation guided by task feedback. Recognizing that omics analysis demands not only high performance but also transparent and verifiable construction principles, we introduce Evolutionary Rule Abstraction to distill generalized logic from optimization trajectories, overcoming the opacity of traditional evolution. Furthermore, to ensure the scientific rigorousness of these discovered principles, we design a Counterfactual Validation Module to strictly verify the causal reliability of abstracted rules. Through this transparent evolutionary paradigm, CEAgent-GSL overcomes the limitations of conventional parametric tuning, and facilitates highly efficient and interpretable GSL, rigorously validated across 11 foundational bioinformatics algorithms on 10 diverse omics datasets. Besides individual task optimization, it can seamlessly integrate domain expertise with task-specific requirements by retrieving and adapting proven logic, thus generating customized rules for novel scenarios.
Zhongyu Xing, Hanwen Luo, Maozu Guo et al.· Proceedings of the 32nd ACM...· 0 citations
Despite the success of Large Language Models (LLMs) in structured query generation, OData—a critical RESTful protocol for enterprise APIs—remains under-researched due to a lack of high-fidelity, execution-validated datasets. To bridge this gap, we introduce O M - NI OD ATA , a framework that generates S YN O-D ATA , the first large-scale OData corpus featuring execution-grounded queries and reasoning traces. Using this corpus, we develop O MNI OD ATA -R1 (1.5B–3B parameters), a family of models that match or surpass frontier proprietary systems, such as GPT-4o and Gemini 3, on realistic industrial benchmarks. Our results demonstrate that the synergy of execution-verified synthetic data and Reinforcement Learning (RL) effectively unlocks the latent reasoning of Small Language Models (SLMs), providing a high-performance, low-latency solution for specialized enterprise query generation. The code and data will be released under an open-source license.
Tao Bai, Zhaochen Li, Hongxin Shao et al.· Proceedings of the 64th Annu...· 0 citations
High quality temporal graph benchmarks with rich semantics and ground-truth anomaly labels are essential for training graph neural networks, yet remain scarce due to privacy constraints and annotation costs. We present SAGA (Synthetic Agentic Graph Architecture), a system for generating large-scale, semantically rich temporal graphs via a four-phase pipeline. Our Skeleton-First, Semantics-Second architecture decouples structure from semantics: (S) an O(1)-per-edge skeleton generator produces power-law graphs; (A) a dispatcher partitions causally ordered time blocks for parallel execution; (G) LLM agents inject domain semantics using RAG-based rule bases across four domains; and (A) a state alignment engine resolves conflicts via temporal replay, yielding anomaly labels as natural byproducts. Unlike structural generators (e.g., LDBC SNB, Kronecker/R-MAT) or purely LLM-based approaches, SAGA achieves structural realism, semantic richness, and automatic anomaly labeling in a unified framework. On a single H100 GPU with vLLM batching, SAGA generates 500,000 temporal edges with controlled anomalies in under 90 minutes, scaling to 100,000 nodes while maintaining clustering coefficients above 0.99. The system supports real-time pipeline visualization, interactive multi-domain tuning (Finance/AML, Network/IDS, Cyber/APT, Transportation), and a CLI for large-scale GPU-based experiments.
This dataset defines a new basis for reproducible and data-driven AI design, advancing the emerging paradigm of LLM-driven AutoML and architectural generalization across modalities and hardware.
Tolgay Atinc Uzun, Waleed Khalid, Saif U Din et al.· 19 citations