The GECCO Benchmark Design Challenge formulates benchmark construction as the problem of maximizing performance differences between optimizers, measured via pairwise rank distances. We approach this as a meta-optimization task and propose an automated framework that generates and refines benchmark suites using large language models (LLMs) and evolutionary search. Candidate functions are synthesized through LLM-driven evolution and evaluated directly on their ability to discriminate between five standard optimizers under a fixed evaluation budget. We further optimise the suite by combinatorial optimization of all generated functions over all considered dimensionalities in the range of [2,80). Our results show that this combined approach substantially increases discriminability, improving scores from approximately 2.04 to 2.58. These findings highlight that effective benchmark design requires not only diverse function structures but also careful control of problem dimensionality, and demonstrate the potential of learning-driven methods for constructing targeted benchmarking suites.
Ananta Shahane, Niki van Stein· Proceedings of the Genetic a...· 0 citations
Large Language Models (LLMs) are increasingly used as generative operators in evolutionary algorithms, yet the role of selection in LLM-assisted systems remains underexplored, particularly when optimizing structured natural language artifacts without ground-truth fitness signals. We introduce EvoHIIT, an LLM-assisted evolutionary framework for the design of High-Intensity Interval Training (HIIT) programs, and study preference-based selection mechanisms. Within an evolutionary loop, EvoHIIT compares four evaluation regimes: (i) scalar fitness based on LLM-predicted physiological and structural proxies, (ii) LLM-based pairwise qualitative selection, (iii) human preference selection, and (iv) hybrid strategies combining automated optimization with periodic human correction. Using schema-constrained HIIT programs as a controlled domain, we analyze convergence dynamics, mutation behavior and alignment with user and expert judgments. Our results show that the choice of selection mechanism substantially shapes search trajectories in semantic solution spaces. While proxy-based scalar fitness enables stable optimization, preference-based selection mitigates specification bias and improves perceived quality. Our findings position selection as a central design dimension in LLM-assisted evolutionary optimization and provide empirical insight into human-aligned evolution of natural language solutions.
Johana Chen, Niki van Stein, Robert Cabri et al.· Proceedings of the Genetic a...· 0 citations