This work forms fairness auditing as a min-max optimization between a computationally unbounded company and a budget-constrained auditor, and demonstrates that increasing audit resources reduces, but does not eliminate, the scope for post-audit manipulation.
Abstract
Fairness audits are increasingly mandated in high-stakes applications such as hiring, lending, and automated decision-making. Recent work has established fundamental impossibility results for black-box fairness auditing, showing that sufficiently expressive models can evade any auditing strategy. We complement these results by quantifying the extent of unavoidable post-audit manipulation under finite audit resources. We formulate fairness auditing as a min-max optimization between a computationally unbounded company and a budget-constrained auditor. We study two auditing regimes: (i) a budgeted auditor that certifies fairness using a fixed-size audit set, and (ii) a budgeted {\alpha}-tolerant auditor that additionally requires the audit set to estimate the fairness of the certified model within an {\alpha} approximation. For both settings, we derive explicit lower bounds on the worst-case post-audit demographic parity deviation as functions of the audit budget, group imbalance, and fairness tolerance. Finally, we empirically illustrate these theoretical limits using simple audit-set construction heuristics with linear and neural network classifiers. Our results demonstrate that increasing audit resources reduces, but does not eliminate, the scope for post-audit manipulation, highlighting fundamental limitations of finite-budget fairness certification.
The Consistency Radius is introduced, a metric that quantifies the maximum distribution shift under which an audit conclusion based on a third-party dataset remains consistent, and a convex relaxation-based optimization method to estimate the radius using only model responses over the audit dataset is proposed.
A novel audit protocol designed to significantly increase the post-audit detectability of manipulations by enabling the auditor to query the model in an oblivious manner and providing theoretical guarantees showing that, under this protocol, a provider attempting to hide unfairness must falsify a significantly larger number of responses.
Augustin Godinot, Sofiane Azogagh, Julien Ferry et al.· 0 citations
Algorithmic credit scoring must satisfy fairness and explanation requirements, yet prevailing predictive-parity criteria assess only outcomes at the decision point. They can therefore overlook whether rejected applicants face unequal burdens in reaching future approval, a phenomenon we call masked inequality. We develop an effort-centric framework that measures an applicant's effort as the minimum weighted cost of feasible changes required to cross the approval boundary. The framework distinguishes feature-independent actions from additive structural shifts that propagate through a causal model and defines parity by comparing average minimum effort across protected groups. We derive tractable local expressions for general differentiable classifiers and exact expressions for logistic regression, embed them in an in-processing fairness objective, and bound changes in portfolio credit risk. The same optimisation yields actionable pathways to approval. Using mortgage data with continuous and discrete features, we find that rejected female applicants require greater effort even when standard predictive-parity criteria are satisfied. Feature-independent regularisation reduces the effort gap by more than 50\% with modest predictive changes. Causal regularisation yields reductions above 90\% at the tested positive penalty weights, but with larger predictive and risk-return trade-offs. Expected and unexpected losses remain broadly stable under feature-independent regularisation and increase under causal regularisation; RAROC declines but remains positive. These results show that effort parity complements predictive fairness by revealing and mitigating hidden barriers to future credit access while making the associated operational trade-offs explicit.
Existing methods for risk-limiting audits typically focus on certifying individual contests. In parliamentary elections, however, the politically relevant outcome is often whether a party has won enough seats to form government, not whether every reported seat outcome is correct. Extending on the work of Mohanty et al. (2019), we formulate the certification of a parliamentary majority as a partial conjunction testing problem: it is enough to verify that the reported winning party truly won at least a majority of its reported seats. Building on the SHANGRLA auditing framework, we construct a sequential audit statistic for the majority outcome by combining seat-level statistics. We then propose adaptive sampling strategies that allocate auditing effort across seats, including variants that learn to avoid spending excessive effort on seats that appear unlikely to have been truly won. Using simulations based on synthetic and real data, from the 2014 Indian Lok Sabha election, we show that auditing the parliamentary majority can substantially reduce the number of ballots inspected (by almost a thousand-fold) compared to certifying every reported winning seat.
Jack Freestone, D. Leung, Damjan Vukcevic· arXiv.org· 0 citations
Algorithmic fairness evaluation commonly assesses AI systems as bounded technical components, abstracting away the organizational context in which they operate. We present, to our knowledge, the first independent end-to-end fairness audit of a semi-automated hiring system operated by Barcelona Activa, a public employment agency using the third-party TalentClue platform for candidate search and shortlisting. We analyze approximately 497,000 candidate-vacancy pipeline entries from September 2017 to September 2022, covering seven pipeline stages that span automated processing, human discretion, candidate data, and employer decisions. Aggregate outcomes across binary genders are statistically indistinguishable, yet this parity masks substantial disparities by salary level, age, and gender identity. Women face adverse impact in mid-salary shortlisting (DIR = 0.786, p<0.001), alongside salary disparities in 15 of 20 sectors and a compounded disadvantage for women aged 46-55 (DIR = 0.77). Non-binary candidates are shortlisted at less than one third the rate of men (DIR = 0.295), although this estimate rests on a small sample (N = 285). Candidates aged 55 and over are entirely absent from the pipeline despite comprising 15.6% of Barcelona's labor force. The gender gap in shortlisting narrows over time, from 6.5 percentage points in 2017 to 1.3 in 2022. The audit further reveals a vendor-deployer information asymmetry: Barcelona Activa lacks access to key information about TalentClue's matching logic and evaluation. Fairness outcomes can thus arise from interactions among automated processing, human discretion, data quality, vendor opacity, and pipeline structure. We build on prior calls for sociotechnical, end-to-end fairness evaluation, showing empirically why model-level assessment alone can be insufficient for understanding fairness in deployed systems.
The AI Bias Firewall (AIBF), a method that audits an applicant tracking system one decision at a time, is presented and a limitation is reported: correcting flagged decisions raises the disparate impact ratio substantially but not to legal parity, because features labeled as merit carry residual proxy correlation.
Jay Barach· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
