Provenance-Aware Quality Control for Heterogeneous Bioactivity Data
Abstract
Background. Machine-learning models for proteins increasingly train on large, pooled collections of laboratory measurements-binding affinities such as IC50, Ki, Kd and EC50 (numbers that describe how tightly a molecule binds its target), togetherwith biophysical readouts from techniques such as surface plasmon resonance (SPR), bio-layer interferometry (BLI) and isothermal titration calorimetry (ITC). These measurements arrive in different units, from different assay formats, fitted with different models, and at different levels of quality. Pooling them without care injects substantial noise. The per-record quality flags that some databases attach are incomplete, and they do not travel with the data once records from several sources are combined. What has been missing is an open, model-agnostic layer that turns raw measurements into records a downstream pipeline can consume directly: unit-normalized, quality-checked, and carrying their own uncertainty and provenance. What we built. biophys_interop is an open Python toolkit and data schema. It maps any supportedmeasurement to a single canonical record spanning 24 measurement types (modalities), converts units to a common standard, and runs a registry of 72 deterministic, method-aware quality-control (QC) rules. Every rule cites a reason from the measurement literature, recommends a concrete action, and returns a simple pass / warn / fail flag together with a calibrated uncertainty. The library is small and has few dependencies. We are deliberate about what our validation does and does not cover: two broadly applicable affinity rules (a plausible-value-range rule and an exact-duplicate rule) are tested at scale against an independent curator’s labels; the remaining method-specific biophysical rules are offered as a literaturegrounded, executable registry-useful and auditable, but not claimed as validated here, because no public database publishes the per-record ground-truth flags that would be needed to score them. Results. Using ChEMBL’s own data_validity_comment and potential_duplicate annotations as an independent ground truth, transparent rules on the standardized record reproduce two of ChEMBL’s three curator flag families on 175,387 held-out records across 30 human targets, with no trained model (value-range rule: Matthews correlation coefficient [MCC] 0.61; duplicate rule: MCC 0.65). The thresholds transfer cleanly to a completely separate set of targets (train on 15, test on 15 disjoint targets; 88,443 held-out records; MCC 0.73 and 0.64). Going beyond the curator, the layer flags 929 internally inconsistent records that ChEMBL did not flag, and a comparison against the independent BindingDB database exposes 38 disagreements of 100-fold or more, 89% of them unmarked by either database. A domain expert who checked a 24-record sample against the primary literature confirmed 16 of 24 (67%) as genuine data errors, including 11 of 12 cross-database conflicts. We reproduce the known cross-assay noise regime reported by Landrum and Riniker, and show that a comparability gate based on assay metadata more than halves the median disagreement between records (0.51-> 0.21 log units). Finally, feeding the calibrated signal into a downstreammutation-effect (ΔΔG) predictor gave no reliable improvement; we report this negative result in full. The toolkit, the rule registry, the cleaned datasets, and one-command reproduction scripts are released openly. Conclusions. Once heterogeneous public measurements share one representation, simple transparent rules can catch data problems-matching expert curators at scale on the rule families we tested, and catching errors those curators missed. The result is a reusable, auditable data layer for protein machine learning, rather than a new predictive model. Keywords: bioactivity data; data curation; quality control; ChEMBL; BindingDB; data standardization; FAIR data; machine learning; interoperability; reproducibility
