Stress-Testing Efficient Responsible-AI Evaluation: When Compute Savings Change Benchmark Conclusions
This work evaluates three dense and mixture-of-experts models on BBQ and BBQ-V under seven conditions spanning batching, quantization, benchmark reduction, and their combinations, and compares accuracy, bias severity and prevalence, reasoning quality, subgroup behavior, subset-membership stability, runtime, and measured GPU energy against a full-benchmark BF16 baseline.
Ahmed El kady, Aravind Narayanan, Rehana Noorani et al.
· 0 citations