Machine Learning-Based Evaluation of Multi-Site MRI Quality Metric Harmonization

Abstract

Multi-site data collection enables the aggregation of large and diverse magnetic resonance imaging (MRI) datasets, which are essential for development of robust machine learning (ML) models in neuroimaging. However, site-related variability introduced by differences in scanner equipment and acquisition protocols (i.e. "batch effects") may confound downstream analyses and obscure meaningful information. Harmonization methods, aim to eliminate this site-induced variability from the data while preserving true biological signals through covariates integrated into the harmonization models. Beside harmonization, MRI quality control is also an essential step in data preparation. However, for multi-site data, image quality metrics (IQMs) designed to capture quality-related properties of the recordings, may also contain site-specific characteristics. Although harmonization methods such as ComBat, are widely used to mitigate batch effects, their impact on IQMs and the role of incorporated covariates remain insufficiently understood. To address these shortcomings, in this study, we evaluate the effects of different batch correction strategies on structural brain MRI IQMs by comparing simple data merging, database-wise standardization, ComBat without and with age and sex included as biological covariates through downstream application of ML models, and by statistical comparison of feature values for validation. We show that both database-wise scaling and harmonization reduce site-related information, however, nonlinear batch effects remain in the data. We also demonstrate that biological information is attenuated if not incorporated into the model as covariates, which in turn reduces harmonization effectiveness. Furthermore, we identify and analyze the most influential IQMs for site, age, and sex prediction across the different data processing strategies.