FIFT: Feature Importance-Guided Fairness Testing for machine learning software
TL;DR
Results indicate that global feature importance, used as an active search signal rather than a post-hoc diagnostic, improves both the effectiveness and the efficiency of individual fairness testing.
Abstract
Fairness in machine learning (ML) is a software quality requirement in high-stakes domains such as healthcare, hiring, and criminal justice, where biased predictions based on protected attributes can harm individuals. Individual fairness testing (IFT) searches for pairs of inputs, known as individual discriminatory instances (IDIs), that differ only in a protected attribute yet receive different classifier predictions, indicating discrimination. Existing search-based IFT methods suffer from three limitations: (1) random feature perturbation that ignores the varying discriminatory influence of features, (2) reliance on computationally expensive per-instance local explanations, and (3) high test redundancy that wastes computational resources. This study introduces FIFT (Feature Importance-Guided Fairness Testing), an evolutionary approach that computes global feature importance once via permutation feature importance and uses the normalised scores to guide the search for IDIs. The approach is motivated by the observation that features with greater predictive influence often provide effective search guidance toward classifier decision boundaries where fairness violations may occur. Rather than treating feature importance as a direct indicator of fairness, FIFT employs it as a computationally efficient heuristic for guiding evolutionary search. FIFT introduces Importance-Guided Hybrid Mutation (IHM), which scales perturbation magnitude inversely to feature importance for influential features while applying random perturbations to less influential ones, thus balancing exploitation and exploration of the input space. Experiments on five benchmark datasets and four ML classifiers show that FIFT detects 20.8%–190.4% more IDIs than the strongest baseline, achieves 2.37×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document}–3.1×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document} higher throughput than local-explanation-based methods, and reduces test redundancy by 84.0%–98.0% relative to other search-based approaches. Retraining with discovered IDIs improves fairness by 28.6%–88.0% with negligible accuracy loss. These results indicate that global feature importance, used as an active search signal rather than a post-hoc diagnostic, improves both the effectiveness and the efficiency of individual fairness testing.
