Deployment-Aware Comparison of Lightweight Bias Mitigation Techniques in Machine Learning Pipelines

Abstract

Bias mitigation techniques for tabular classification are widely studied for their fairness and accuracy properties, but their deployment cost—including latency, training overhead, and implementation complexity—is reported less consistently. We present a deployment-aware empirical comparison of three lightweight bias mitigation methods (Reweighing, Disparate Impact Remover, and Equalized Odds post-processing) against an unmitigated baseline, evaluated across three benchmark datasets (Adult, COMPAS, German Credit) and two classifiers (Logistic Regression and Random Forest). We report performance, fairness, and operational cost across 300 runs total in the main matrix (240 runs with default AIF360 DIR + 60 runs with our Disparate Impact Remover sub-protocol that uses a train-only fitting procedure to avoid the feature-distribution leakage we observed in AIF360’s standard transductive DisparateImpactRemover API). Results show that the best method is not constant: Equalized Odds reliably reduces equalized-odds disparity (<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula>EOD<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula>) across all settings at a 2–10 percentage-point accuracy cost; Reweighing reduces demographic-parity disparity (<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula>SPD<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula>) substantially on Logistic Regression (47–80% reduction) but barely affects Random Forest on the larger datasets (4–9% reduction on Adult and COMPAS); and under the corrected train-only protocol, Disparate Impact Remover does not consistently reduce <inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula>SPD<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula> (<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula>SPD<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula> reduction is below 4% in 3 of 6 configurations and <inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula>SPD<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula> increases by 3–54% in the remaining 3 configurations), at an accuracy cost within ±1 percentage point and batch-256 inference-latency multipliers of 12–<inline-formula> <tex-math notation="LaTeX">$49\times $ </tex-math></inline-formula> on Logistic Regression and 1.3–<inline-formula> <tex-math notation="LaTeX">$2.4\times $ </tex-math></inline-formula> on Random Forest. Under classifier hyperparameter tuning on the validation fold, the Reweighing<inline-formula> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula>Random-Forest gap closes markedly: tuned RF (max_depth= 10, class_weight=’balanced’ most often selected) achieves 30-49% <inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula>SPD<inline-formula> <tex-math notation="LaTeX">$\vert $ </tex-math></inline-formula> reductions with Reweighing, indicating that the RF gap under default hyperparameters is an artifact of the defaults rather than of Reweighing itself. We synthesize these results into a model- and constraint-conditional decision framework, and we report the AIF360 transductive-DIR leakage as a methodological observation that subsequent empirical fairness studies should address. Code, data-loading procedures, and the train-only DIR module are available at: <uri>https://github.com/shiva88/fairness-deployment-comparison</uri>