Not All Problems Are Equal: Weighted Performance Profiles For Many-Objective Optimization
TL;DR
By employing a difficulty-aware weighting scheme, the approach biases aggregation toward higher-dimensional instances, enabling a more discriminative assessment of scalability, robustness, and performance.
Abstract
To ensure empirical evaluation of multi- and many-objective evolutionary algorithms, researchers perform benchmarking across test problems and algorithms. Due to the volume of performance data and the heterogeneity of problem characteristics, analyzing results becomes complex and prone to misinterpretation. Performance profiles have proven effective for visualizing and interpreting such results; however, they do not account for the relative difficulty or importance of individual problems and may overweight easy or less informative cases, potentially obscuring distinctions between algorithm performance. In this work, we address this limitation by extending the classical performance profile approach with a difficulty-aware weighting scheme that emphasizes more challenging problems. Weights can be assigned either a priori, based on problem characteristics such as the number of objectives or decision variables, or a posteriori, based on computational effort. We define and prove key mathematical properties of classical performance profiles, including local and global stability, and show that these properties extend to the proposed weighted formulation. By employing a difficulty-aware weighting scheme, the approach biases aggregation toward higher-dimensional instances, enabling a more discriminative assessment of scalability, robustness, and performance. The advantages of the weighted approach are demonstrated through experiments with algorithms applied to problem sets with numbers of objectives.