FairSelect provides a practical framework for systematically identifying fairness strategies that improve subgroup equity while preserving model performance in clinical machine learning.
Abstract
Algorithmic fairness methods are increasingly used to identify and mitigate bias in machine learning models, yet most approaches are evaluated in isolation and along single demographic axes. This limits practical guidance for selecting fairness strategies, where disparities may arise across intersectional subgroups and across multiple stages of the modeling lifecycle. This work presents FairSelect, a toolkit for systematically evaluating fairness mitigation strategies applied individually and in combination across preprocessing, inprocessing, and postprocessing stages. FairSelect supports multiple model architectures, intersectional subgroup evaluation, and comparison of fairness utility tradeoffs across baseline, single method, and multi level configurations. The framework was validated using synthetic clinical datasets designed to represent specific bias mechanisms and a real-world replication of two-year stroke risk prediction among patients with atrial fibrillation. Synthetic experiments showed that targeted fairness methods generally reduced intended subgroup disparities, while combined strategies produced larger average fairness improvements with modest utility tradeoffs. In the clinical prediction task, mitigation effects were highly variable, with some combinations improving both fairness and predictive performance while others were ineffective or counterproductive. These findings demonstrate that fairness interventions interact in nonadditive and context dependent ways. FairSelect provides a practical framework for systematically identifying fairness strategies that improve subgroup equity while preserving model performance in clinical machine learning.
It is demonstrated that while DP alone can degrade both utility and fairness, applying fairness interventions can partially restore equitable outcomes, and post-processing methods tend to provide more stable fairness–utility trade-offs across privacy budgets and synthesizers.
Vinícius Gabriel Angelozzi, Héber H. Arcolezi· Proceedings on Privacy Enhan...· 1 citation· ⚡1
As regulatory requirements increasingly shape automated lending decisions, fairness remains a critical challenge in high-stakes domains, particularly credit scoring. Although artificial intelligence models can achieve strong predictive performance, they may also reproduce biased outcomes that reduce financial inclusion or transfer harm to overlooked protected groups. Existing fairness interventions commonly operate at a single stage of the decision-making pipeline, despite bias often propagating across representational and decision layers. This study proposes iCert-Fair, a two-layer framework for technical fairness assessment and harm recovery in credit scoring. The first layer adopts a fairness-through-explainability paradigm, using SHAP-based explanations to identify direct and proxy dependence on protected attributes and guide structural dataset repair, while the second layer applies targeted threshold-policy adjustments to recover residual harm while preserving decision utility. Experiments on the German and Taiwanese credit datasets show that fairness gains are model- and dataset-specific and may be collective, concentrated, transferred, or recovered unevenly across protected attributes. The direct comparison with representative pre-processing, in-processing, and post-processing methods revealed that baseline methods targeting one protected attribute at a time frequently transferred residual harm to other monitored attributes. In contrast, the fairness-focused recommendations generated by iCert-Fair achieved larger collective fairness improvements across all considered protected attributes while avoiding residual harm. These gains were obtained while preserving predictive utility on the German dataset and with utility degradation remaining below 5% across the evaluated performance metrics on the Taiwanese dataset, alongside consistently lower false-negative risk. The empirical findings support the use of complementary structural and policy-level interventions and demonstrate the importance of jointly evaluating aggregate disparity, worst-case attribute-level harm, cross-attribute transfer, and predictive utility.
Machine learning increasingly drives decisions in domains such as finance and healthcare, where ethical considerations, such as fairness, are central. In such contexts, ensuring fairness is essential, especially when decisions impact individuals and social groups. Federated learning (FL) provides a decentralized training paradigm, yet client heterogeneity and demographic imbalance can amplify disparities across subpopulations. Existing fairness-aware FL methods remain limited, often focusing on group fairness in binary classification and lacking explicit control over the trade-off between fairness and predictive performance. We introduce FedFairLAB, a FL method that enforces group, intersectional, and multiclass fairness simultaneously at both the local and global levels. A tunable performance budget allows practitioners to control how much predictive performance can be sacrificed to improve fairness. Experiments on six real-world datasets show that FedFairLAB substantially improves fairness while keeping models accurate and usable in realistic FL settings.
Michele Fontana, Francesca Naretto, A. Monreale· Data mining and knowledge di...· 0 citations
The results show that ML data selection can hurt model fairness in a non-negligible number of cases, and compromise model utility in more than half of the cases, and provide interesting research directions for utility- and fairness-aware ML data selection.
Nawel Benarba, Zeyang Kong, Sara Bouchenak· IEEE Access· 1 citation
The growing use of machine learning in high-stakes domains raises concerns about fairness. The role of optimization metrics in shaping these outcomes remains underexplored. Using a controlled setup, this study investigates how seven performance metrics used for hyperparameter tuning and model selection affect fairness outcomes across five benchmark datasets. Results show that metrics are not neutral: recall-based optimization yields higher disparities, while precision and specificity lead to more balanced outcomes, with PR-AUC showing intermediate behavior. Overall, metric choice influences fairness, but outcomes are largely driven by dataset characteristics, with optimization redistributing errors rather than eliminating bias.
Bianca Matos de Barros, Diego Dimer Rodrigues, G. Oliveira et al.· Anais do LIII Seminário Inte...· 0 citations
Abstract Background Fairness evaluation is essential for trustworthy clinical risk prediction. However, existing fairness-oriented discrimination metrics either ignore cross-group comparisons or rely on exhaustive pairwise evaluations, making them difficult to interpret and impractical for model selection. Objective This study aimed to develop and evaluate novel fairness-oriented discrimination metrics for clinical risk prediction that address limitations of within-group and pairwise cross-group approaches. Methods We examined theoretical properties of existing U-statistic–based metrics, including concordance index (CI) and area under the receiver operating characteristic curve (AUC), when applied to subgroups. We highlighted the distinction between within-group discrimination (ranking within a subgroup) and group-level discrimination (ranking relative to the broader population). Building on this framework, we proposed group-level extensions of the CI and AUC that summarize subgroup-specific performance in a single interpretable measure. We then applied these metrics to the PREVENT (Predicting Risk of Cardiovascular Disease Events) equation, a recently developed model for atherosclerotic cardiovascular disease. Results The traditional subgroup-specific CI and AUC captured within-group but not group-level discrimination, obscuring inequities in clinical decision-making. Existing cross-group approaches (eg, the xCI and xAUC metrics) addressed this limitation but became computationally and interpretively burdensome with multiple subgroups due to pairwise comparisons. Our proposed metrics provided a streamlined alternative, yielding 1 summary statistic per subgroup while retaining sensitivity to cross-group ranking disparities. Applied to PREVENT, these metrics revealed differences in subgroup performance not apparent from within-group evaluations. Conclusions By distinguishing between within-group and group-level discrimination, our framework clarifies a common source of misinterpretation in fairness evaluation. The proposed group-level extensions of the CI and AUC provide practical, interpretable tools for evaluating fairness in clinical prediction models, enabling more transparent and equitable risk assessment.
Haoyuan Wang, Chuan Hong, Michael J. Pencina et al.· JMIR AI· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
