115 Assessing and assuring equity for artificial intelligence in neuro-oncology with Fairboard
Abstract
Equity, the absence of unfair, avoidable or remediable differences among groups, is a WHO policy tenet yet rarely measured in medicine. Neuro-oncology is no exception: its models are driven by population-level data with little attention to distinctive subpopulations. We propose an open-source platform to quantify and improve the equity of statistical models in neuro-oncology and beyond. We developed a web-based, no-code application for benchmarking model equity (including AI) in medical research, leveraging healthcare economics, Bayesian and frequentist statistics, neuroimaging, and machine learning. We demonstrate its utility in an evaluation of 648 adult glioma patients across 20 open-source tumour segmentation models (n = 12,960 segmentations), spanning >6 years of machine learning research. Higher mean Dice was strongly associated with lower Gini index, i.e., higher overall performance models conveyed less overall performance inequity (Spearman’s rho = -0.956, all p < 0.0001). However, composite performance-equity league tables showed substantial rank reshuffling relative to performance-only rankings, with half of the models shifting by 4 or more positions, demonstrating that equity assessment often alters model selection. 17/20 models conferred a performance advantage to males; only 3 showed a marginal female advantage. Age effects were tissue-dependent: younger patients achieved better whole-tumour but worse tumour-core and enhancing-tumour segmentation. The largest disparities arose from the WHO 2021 diagnosis (IDH-mutant astrocytoma tumour core Dice 0.59 vs GBM IDH-wildtype 0.91) and extent of resection, where gross total resection (GTR) outperformed subtotal (STR) across all tissue classes (all p < 0.003). Our open-source platform, deployable on any device via a web browser, enables any user to interrogate the equitable benefit of models across patient subgroups, positioning neuro-oncology to lead in maximising healthcare equity.