Jul 2026· Proceedings of the Genetic and Evolutionary Computation Conference Companion· pp. 83-84· 0 citations· 11 references
TL;DR
BONO-Bench is presented, a recently proposed problem generator and benchmark set for bi-objective numerical optimization that enables best practices for empirical runtime analysis of optimizers using reference solutions that can be approximated to an arbitrary degree, resulting in precise target values for the hypervolume and exact R2 indicators.
Abstract
The evaluation of heuristic optimizers on test problems, better known as benchmarking, is a cornerstone of research in multiobjective optimization. However, many frequently used test problems either feature a limited degree of optimization challenges or have poorly understood reference solutions. Here, we present an overview of BONO-Bench [5], a recently proposed problem generator and benchmark set for bi-objective numerical optimization. Building on convex-quadratic problems, it features diverse challenges ranging from different levels of conditioning, shapes of Pareto set and front as well as plateaus to different structured and unstructured multimodality patterns. Furthermore, we enable best practices for empirical runtime analysis of optimizers using reference solutions that can be approximated to an arbitrary degree, resulting in precise target values for the hypervolume and exact R2 indicators.
Symbolic regression (SR) is commonly benchmarked using relative Pareto analysis, where an algorithm is evaluated according to whether it dominates the other methods included in the comparison. While useful, this perspective does not reveal the true attainable limits of short, interpretable expressions, and conclusions may change depending on which competing methods are selected. In this work, we advocate benchmarking with absolute Pareto-optimal (APO) fronts instead. We construct APO fronts for 34 datasets from SRBench by exhaustively searching over short symbolic expressions and by fitting numerical constants with eight commonly used local optimization methods. The resulting fronts provide dataset-specific baselines for the best accuracy-complexity trade-offs attainable within a fixed primitive set. Comparing against the SR methods reported in SRBench shows that many current algorithms remain far from these fronts, especially in the regime of short expressions. We also find that the final fronts are relatively stable across numerical optimizers, suggesting that structural search is a more important bottleneck than constant fitting. These APO fronts provide a reusable benchmarking asset for the evolutionary computation community and a more stable reference point for measuring progress in SR.
Kei Sen Fong, M. Motani· Proceedings of the Genetic a...· 0 citations
We propose a couple of novel tools for benchmarking optimization algorithms which possibly converge to different solutions on a test set: the quality profiles and the test set profiles. Their aim is to assess and compare algorithms in terms of quality (i.e. value of the objective function) of the obtained solutions, as well as to assess the consistency of the test set. A key distinguishing feature of the quality profiles we propose is its comparative deterministic procedure that emphasizes the accuracy of the solution, rather than the computational burden of solvers. In this regard, several test set--dependent approaches for both comparing and ranking algorithms have already been proposed in the literature, representing widely used benchmarking procedures. We believe that the joint use of such procedures, along with the novel quality profiles detailed here, should enhance the benchmarking process, in all those cases where the comparison encompasses exact methods as well as heuristics. Moreover, the literature on numerical optimization seems to have paid less attention, in the last decade, to determining how appropriate a test set used for benchmarking the selected solvers may be. This motivates the introduction of test set profiles, which assess the appropriateness of a test set and represent the flip side of evaluating the robustness of the solvers on that test set. This paper also includes extensive numerical experiments, showing the usefulness of quality profiles in both smooth and nonsmooth (derivative--free) optimization, along with the reference to a MATLAB code for plotting quality profiles and test set profiles.
The Pareto set and Pareto front of a continuous multiobjective optimization problem typically contain infinitely many solutions. Since dealing with infinite sets is impractical, they are commonly approximated by finite reference sets. In this way, the performance of multiobjective optimization algorithms can be assessed using quality indicators. However, the selection of a reference set depends on the intended goal, such as achieving a uniform distribution of solutions along the Pareto set or Pareto front, or optimizing a specific quality indicator. In this paper, we investigate and compare six strategies for reference set construction in constrained bi-objective problems from a recently proposed test problem generator. The approaches are evaluated with respect to multiple quality indicators and computational cost. The experiments are performed on test problems with irregular Pareto sets and fronts. From these experiments, we conclude that the choice of strategy should depend on the intended goal, with the approach that aims to maximize the hypervolume indicator value standing out as the best trade-off in terms of speed and indicator accuracies.
Luka Opravš, D. Brockhoff, T. Tušar· Annual Conference on Genetic...· 0 citations
By employing a difficulty-aware weighting scheme, the approach biases aggregation toward higher-dimensional instances, enabling a more discriminative assessment of scalability, robustness, and performance.
Regina C. L. C. de Sousa, Dênis E. C. Vargas, Elizabeth F. Wanner et al.· Annual Conference on Genetic...· 0 citations
Diversity optimization is the class of optimization problems in which we aim to find a diverse set of good solutions. One of the frequently-used approaches to solve such problems is evolutionary diversity optimization (EDO). In this paper, we analyze EDO on a three-objective function LOTZk, which is a modification of the two-objective benchmark function (LeadingOnes, TrailingZeros). We prove that the GSEMO computes a set of all Pareto-optimal solutions in O(kn3) expected iterations. We also analyze the runtime of the GSEMOD algorithm (a modification of the GSEMO for diversity optimization) until it finds a population with the best possible diversity for two different diversity measures: the total imbalance and the sorted imbalances vector. For the first measure we show that the GSEMOD optimizes it in O(kn2 log(n)) expected iterations (which is asymptotically faster than the upper bound on the runtime until it finds a Pareto-optimal population), and for the second measure we show an upper bound of O(k2n3 log(n)) expected iterations. The complementary empirical study shows a very similar behavior for both diversity measures. The results of experiments suggest that our bounds for the total imbalance measure are tight, while the bounds for the imbalances vector are too pessimistic. This paper summarizes the work Denis Antipov, Aneta Neumann, Frank Neumann and Andrew M. Sutton: Runtime Analysis of Evolutionary Diversity Optimization on the Multi-objective (LeadingOnes, TrailingZeros) Problem. Evolutionary Computation, 1–23, 2025. [2].
D. Antipov, Aneta Neumann, Frank Neumann et al.· Proceedings of the Genetic a...· 0 citations
A rigorous runtime analysis of NSGA-III on the many-objective OneJumpZeroJump benchmark, providing runtime bounds where the number of objectives is constant and a stochastic population update provably guarantees a speedup of order Θ((k/b)k-1) in the runtime where b > 0 is a constant.
Andre Opris· GECCO Companion· 11 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
