Skip to content
Log in
Sponsored AI Networking Cookbook: Practical recipes for AI-assisted network automation and development AI Networking Cookbook: Practical recipes for AI-assisted network automation and development
Open access

Search-Based Test Case Generation for Unfairness Detection in Multi-Layer Perceptron-Based Systems: A Genetic Algorithm Approach and Empirical Evaluation

Roa Alharbi Noureddine Abbadeni
2026 · IEEE Access · Vol 14, pp. 120661-120683 · 0 citations · 48 references
Computer Science

TL;DR

A Genetic Algorithm–based Test Case Generation (GATCG) framework that combines knowledge-based reasoning with multi-objective evolutionary optimization to reveal unfair behavior dynamically is proposed that extends beyond single-metric detection to include automatic and adaptable fairness testing.

Abstract

Fairness in machine learning (ML) and deep learning (DL) systems is becoming increasingly important because algorithmic decisions start to have a considerable influence on the governance of some high-stakes domains, making unfairness detection and testing crucial. Despite notable progress, existing automated fairness testing approaches remain limited by static fairness definitions, single-objective optimization, and restricted adaptability across models and datasets. This study addresses these limitations by proposing a Genetic Algorithm–based Test Case Generation (GATCG) framework that combines knowledge-based reasoning with multi-objective evolutionary optimization to reveal unfair behavior dynamically. Our approach comprises a Fairness Knowledge Base (FKB) that formally encodes fairness concepts through logical rules and semantic networks, thereby enabling explicit analysis of the relationships between protected attributes, fairness metrics, and constraints. This knowledge-driven foundation is tightly integrated with a multi-objective genetic algorithm that evolves test cases via selection, crossover, and mutation while simultaneously optimizing disparate impact, disparate treatment, and disparate mistreatment. By combining structured fairness reasoning with evolutionary search, the proposed approach extends beyond single-metric detection to include automatic and adaptable fairness testing. Experiments conducted on benchmark datasets (Adult Income and German Credit) using a Multi-Layer Perceptron (MLP) classifier as the underlying model demonstrate that GATCG achieves lower discrimination sample search time and higher detection success rates compared to baseline approaches (ExpGA, AEQUITAS, SG and ADF). Prior to introducing the uniqueness-tracking mechanism, GATCG demonstrated strong effectiveness by reducing discriminatory sample search time (DSS) by 35%, increasing the number of discovered discriminatory samples (DSN) by 22%, expanding test sample generation (TSN) by 8%, and improving the success rate (SUR) by 9% compared to baseline approaches. To provide a more accurate assessment of the approach, a uniqueness-tracking mechanism and a one-per-generation rule were subsequently incorporated to eliminate duplicate discriminatory discoveries. Under this enhanced evaluation setting, GATCG achieved overall success rates (SUR) of 61.85% on the Adult Income dataset and 68.17% on the German Credit dataset. The results indicate that GATCG efficiently directs the search toward unique discrimination-prone regions of the input space, leading to improved detection speed, more effective identification of biased samples, higher-quality test generation, and strong discriminatory sample discovery performance. Finally, we conduct a controlled experiment that demonstrates GATCG’s superiority over THEMIS and manual testing with respect to unfairness detection effectiveness, usability, coverage, and testing efficiency.

DOI Semantic Scholar
Read PDF

Similar papers

#software testing Open access Aug 2026

FIFT: Feature Importance-Guided Fairness Testing for machine learning software

Results indicate that global feature importance, used as an active search signal rather than a post-hoc diagnostic, improves both the effectiveness and the efficiency of individual fairness testing.

H. Mamman, Abdullateef Oluwagbemiga Balogun, Mustapha Maidawa et al. · 0 citations
Open access Aug 2026

RL–LLMfuzzer: reinforcement learning–guided dual–model differential fuzzing for compiler testing

RL-LLMfuzzer, a reinforcement learning and LLM-based differential fuzzing framework that has successfully unearthed 28 officially confirmed vulnerabilities in GCC and LLVM/Clang, establishing a highly efficient and scalable paradigm for LLM-driven compiler validation.

Donghui Li, Yingying Li, Bo Zhao et al. · 0 citations
Jul 2026

HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks

Heterogeneity-Aware Reward-Guided Optimization is proposed, which introduces per-response importance weighting via confidence-modulated advantage: computing a discrimination signal from group-level reward contrast and a confidence signal from reference model log-probabilities, then modulating the advantage before computing per-response weights, without requiring task-type labels.

Tiangang Li, Xiangbo Tian · 0 citations
Open access Jul 2026

From fair graphs to fair data: a DAG-based approach to mitigating bias in AI systems

Ensuring fairness when training machine learning (ML) models remains a critical challenge, particularly when biases are embedded in the underlying data. This paper presents a fairness-aware graph structure learning framework demonstrating how learning fair graphs leads to fairer data for ML training and, consequently, fairer artificial intelligence (AI) decisioning based on such models. Our method incorporates a fairness regularization term into score-based structure learning algorithms, guiding the search toward graph structures that minimize discriminatory pathways while preserving statistical relationships. The learned fair graph structures enable the generation of synthetic datasets with mitigated biases, which can be used to train diverse ML models. This modification is non-trivial, as structure learning algorithms rely on local search strategies, while fairness is a global property that depends on the entire graph structure. Our framework is highly adaptable, compatible with various structure learning algorithms, and seamlessly incorporates different fairness metrics to meet specific contextual needs. Extensive experiments on both real-world and synthetic datasets demonstrate that our approach significantly improves fairness while maintaining competitive predictive performance, offering an interpretable and versatile solution for mitigating bias in AI systems.

V. Jiang, Gustavo Batista, Michael Bain · 0 citations
Book Open access Aug 2026

MORE-FE: Multi-Operator and Reinforcement Learning-Enhanced Evolution for LLM Feature Engineering

Traditional automated feature engineering (AFE) minimizes human intervention but often neglects semantic relationships among original features, resulting in redundant or uninterpretable transformations. While recent methods leverage the domain knowledge and reasoning capabilities of Large Language Models (LLMs), they typically constrain the search space via predefined mathematical operators or suffer from poor exploration-exploitation balance due to exclusive reliance on validation feedback. To address these limitations, we propose MORE-FE, an evolutionary AFE framework that integrates multi-operator exploration with Reinforcement Learning with Verifiable Rewards (RLVR). MORE-FE treats feature transformation programs as hypotheses and evolves them using evolutionary operators that explore semantically coherent and logically structured features through various prompting strategies. Moreover, it employs RLVR to align LLM reasoning with structured evolutionary exploration using a composite reward that balances quality and diversity. Extensive experiments on multiple classification and regression datasets demonstrate that MORE-FE consistently outperforms state-of-the-art AFE methods, highlighting the effectiveness of balancing quality and diversity in feature engineering.

Chang-Yu Chao, Bryan Andersen, Xiao-Xi Tan et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.