This paper moves beyond point estimation and discusses frequentist and Bayesian approaches to uncertainty quantification for fair machine learning, with practical examples and implications for simulated and real data.
Abstract
Machine learning models are widely used in clinical applications, social media, law enforcement and critical infrastructure. Verifying whether their outputs are biased against disadvantaged groups or individuals is crucial to ensuring they are fair and allowing their use in such settings. A rigorous risk assessment of possible fairness violations requires quantifying the uncertainty associated with selecting and estimating such models. Yet, this is rarely done in the literature, which focuses on identifying a single model with a suitable trade-off between predictive accuracy and fairness. In this paper, we move beyond point estimation and discuss frequentist and Bayesian approaches to uncertainty quantification for fair machine learning, with practical examples and implications for simulated and real data.
An example of its application in the legal field that explores the existing biases in models today is presented and it is hypothesized that the fairness criteria being evaluated cannot all three be satisfied simultaneously, leading to a machine learning model that remains unfair.
Keshav Pillutla, Kevin Fry· Journal of emerging investig...· 0 citations
Results indicate that global feature importance, used as an active search signal rather than a post-hoc diagnostic, improves both the effectiveness and the efficiency of individual fairness testing.
H. Mamman, Abdullateef Oluwagbemiga Balogun, Mustapha Maidawa et al.· Journal of King Saud Univers...· 0 citations
Algorithms are increasingly being used to help automate and improve data-driven decisions, but care must be taken to prevent such algorithms from learning discriminatory patterns from historical data and perpetuating their biases. Statistical notions of fairness aim to mitigate either a model's disparate impact on disa...
Uncertainty estimate is redefined as essential to machine learning performance rather than as a byproduct, and a systematic research program centered on unified theoretical frameworks, uniform evaluation procedures, domain-sensitive calibration, and scalable uncertainty-aware structures is suggested.
Precious Chidum Amadi· International Journal of Com...· 0 citations
We consider the application of automated machine learning models in a real credit lending platform, and empirically study whether the trade-off between fairness and profitability can be improved by means of fairness processing methods. Fairness is evaluated along the principle of Separation, while profitability is comp...
Paolo Giudici, Branka Hadji Misheva, Giacomo Piana· AI and Ethics· 0 citations
This paper develops theoretical bounds on the unfairness and expected loss at deployment, and derives sufficient conditions under which fairness and accuracy can be perfectly transferred via invariant representation learning, and designs a learning algorithm such that fair ML models learned with training data still hav...
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
