The results show that ML data selection can hurt model fairness in a non-negligible number of cases, and compromise model utility in more than half of the cases, and provide interesting research directions for utility- and fairness-aware ML data selection.
Abstract
With the continuous growth in data volumes, efficient data selection has become critical for scaling machine learning (ML) classifiers. Although the existing methods aim to accelerate training without compromising accuracy, their impact on fairness and broader utility remains largely unexplored. This paper presents the first quantitative and qualitative evaluation of these effects through a large-scale empirical analysis comprising 24,395 experiments across 9 real-world datasets, 7 ML models, and several state-of-the-art data selection methods, conducted using ELAPSE, our open-source evaluation framework. Our analysis reveals several key findings and non-trivial observations. The results show that ML data selection can hurt model fairness in a non-negligible number of cases, and compromise model utility in more than half of the cases. These effects differ across datasets, with no single method consistently outperforming the others, and in nearly one-third of the cases, either utility or fairness must be prioritized. Surprisingly, combining data selection with classical bias mitigation increases model unfairness and often degrades model utility. These findings provide interesting research directions for utility- and fairness-aware ML data selection.
Machine learning has become increasingly prevalent in the world as a result of a combination of factors, notably computing breakthroughs and increased data availability. The three most common criteria of fairness in machine learning, despite their common sense and moral appeal, are often mutually exclusive. This frequently presents a challenge and the need to prioritize one criterion over the others. Specifically, the paper highlights the general overview of a machine learning model and presents an example of its application in the legal field that explores the existing biases in models today. We then delve into the three main criteria of machine learning fairness: independence (fairness based on outcomes being unrelated to different characteristics), separation (fairness based on equal error rates across groups), and sufficiency (fairness based on predictions being equally reliable for all groups). In the present experiment, we aimed to investigate the mutual exclusivity of these three machine-learning fairness criteria. We hypothesized that the fairness criteria being evaluated cannot all three be satisfied simultaneously, leading to a machine learning model that remains unfair. The results demonstrated that no threshold in the model simultaneously satisfied independence, separation, and sufficiency, highlighting the limitations of machine learning models that pose various issues across different sectors.
Unknown authors· Journal of emerging investig...· 0 citations
The growing use of machine learning in high-stakes domains raises concerns about fairness. The role of optimization metrics in shaping these outcomes remains underexplored. Using a controlled setup, this study investigates how seven performance metrics used for hyperparameter tuning and model selection affect fairness outcomes across five benchmark datasets. Results show that metrics are not neutral: recall-based optimization yields higher disparities, while precision and specificity lead to more balanced outcomes, with PR-AUC showing intermediate behavior. Overall, metric choice influences fairness, but outcomes are largely driven by dataset characteristics, with optimization redistributing errors rather than eliminating bias.
Bianca Matos de Barros, Diego Dimer Rodrigues, G. Oliveira et al.· Anais do LIII Seminário Inte...· 0 citations
As machine learning (ML) algorithms are increasingly used in high-stakes applications, concerns have arisen that they may be biased against certain social groups. Although many approaches have been proposed to make ML models fair, they typically rely on the assumption that data distributions in training and deployment are identical. Unfortunately, this is commonly violated in practice and a model that is fair during training may lead to an unexpected outcome during its deployment. Although the problem of designing robust ML models under dataset shifts has been widely studied, most existing works focus only on the transfer of accuracy. In this paper, we study the transfer of both fairness and accuracy under domain generalization where the data at test time may be sampled from never-before-seen domains. We first develop theoretical bounds on the unfairness and expected loss at deployment, and then derive sufficient conditions under which fairness and accuracy can be perfectly transferred via invariant representation learning. Guided by this, we design a learning algorithm such that fair ML models learned with training data still have high fairness and accuracy when deployment environments change. Experiments on real-world data validate the proposed algorithm. Model implementation is available at https://github.com/pth1993/FATDM.
This work introduces a method to lower-bound the discrepancy of a classifier: a quantity that jointly captures inaccuracy and unfairness, and develops a computationally efficient procedure for calculating the tightest possible lower bound on the classifier’s discrepancy.
Federated learning (FL) is a promising approach for training distributed machine learning models while preserving clients’ data privacy. However, in real-world FL systems, data are often not independent and identically distributed (non-IID). This heterogeneity can slow convergence, degrade model performance, and increase client drift. To address these challenges, numerous methods have been proposed to mitigate non-IID data effects by optimizing client selection, local training, and model aggregation strategies. Despite their effectiveness in improving performance and efficiency, these methods rarely consider fairness across clients. Improving global accuracy does not guarantee balanced participation, influence, or outcomes, which may lead to biased model behavior across clients. In this survey, we review existing non-IID mitigation methods in FL from a fairness perspective and provide a systematic analysis of their implicit impact on client participation and influence. Unlike prior surveys that treat fairness as a separate research direction, this work analyzes how these methods designed for non-IID mitigation implicitly shape fairness outcomes across clients. Our taxonomy classifies existing methods into three categories—fairness-aware, semi-fairness-aware, and fairness-unaware—based on their design strategies for client selection and model aggregation. Using this taxonomy, we analyze the advantages, trade-offs, and limitations of each category and highlight that mitigating non-IID data does not guarantee fairness across clients. Finally, we identify open challenges and outline future directions, including system-level FL design that jointly considers non-IID mitigation and fairness and the development of standardized fairness evaluation metrics. Overall, this survey aims to provide a structured perspective on the relationship between non-IID mitigation and fairness and support the development of more balanced and scalable FL systems under non-IID conditions.
Mohannad Alsofyani, Isra Al-Turaiki, H. Mathkour· Electronics· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
