It is demonstrated that both dataset quality and data quantity contribute to model performance and, notably, that retaining molecules with incomplete stereochemical information can outperform their removal when the resulting gain in data quantity offsets the quality penalty.
Abstract
High-quality molecular datasets are essential for reliable machine learning in cheminformatics and bioinformatics, yet dataset quality is rarely assessed systematically and its relationship with downstream model performance remains poorly understood. Here, we present MolJam, an open-source frame-work for quantitative assessment of molecular dataset quality across five dimensions—structural integrity, data quality, experimental information quality, chemical space coverage, and data distribution—using 12 standardized metrics. Application of MolJam to 11 MoleculeNet and 8 ChEMBL-derived datasets revealed widespread and heterogeneous quality issues, including undefined stereochemistry in up to 70.72% of molecules, inconsistent molecular representations, and contradictory labels. We next asked whether improving these quality metrics necessarily improves machine learning performance. Refinement of the ESOL and Lipophilicity datasets increased their MolJam quality scores but produced mixed effects on predictive performance, suggesting a competing influence of reduced dataset size. Controlled ablation experiments further demonstrated that both dataset quality and data quantity contribute to model performance and, notably, that retaining molecules with incomplete stereochemical information can outperform their removal when the resulting gain in data quantity offsets the quality penalty. Thus, molecular dataset curation cannot be reduced to maximizing data cleanliness alone but requires balancing multiple dimensions of data quality against information loss. MolJam provides a standardized framework for diagnosing molecular dataset limitations, comparing benchmark quality, and quantitatively evaluating how data curation decisions influence downstream machine learning.
Abstract Motivation Molecular scoring is a popular approach for studying pathway-level functional alterations with omics data. Using molecular scores for tasks such as single-sample molecular characterisation, phenotype prediction or disease stratification has several advantages compared to using omics data directly. Molecular scores provide biological interpretability and are more generalisable across datasets, facilitating data integration and machine learning applications. However, numerous scoring methods are available through different software packages, and currently there is a lack of tools to easily use these scores for model training and prediction. Results We developed pathMED, an R/Bioconductor package that unifies various scoring methods in a simple framework. Furthermore, pathMED also contains a machine learning module to train and test models that use the calculated molecular scores to predict clinical outcomes. We demonstrate some of its potential applications in three use cases using public omics data. We showed the generalisability of machine learning models trained on transcriptomic scores in predicting clinical outcomes when deploying on proteomic scores. We also demonstrated the application of transcriptomics scores in predicting breast cancer treatment response and identifying pathways strongly associated to tumour biology and treatment response. Finally, we demonstrated the benefit of integrating a novel gene set dissection step into the analysis pipeline to resolve disease heterogeneity at the pathway level. Availability PathMED is freely available in the Bioconductor repository (https://bioconductor.org/packages/release/bioc/html/pathMED.html). Code to reproduce the analyses is publicly available at https://github.com/GENyO-BioInformatics/pathMED_article.
Jordi Martorell-Marugán, Ivan Ellson, R. López-Domínguez et al.· Bioinform.· 0 citations
This tutorial provides a comprehensive, end-to-end workflow from raw data to deployed models,icitly designed for environmental chemists with limited prior experience in ML modeling while also providing practical guidance for other users seeking to strengthen their modeling workflows.
Kai Zhang, Yushu Cheng, Hai-Ping Ai et al.· ACS Environmental Au· 0 citations
This review provides a systematic overview of recent advances in SSL-based molecular property prediction and analyzes how multimodal molecular representation learning by integrating sequence, graph, three-dimensional structure, and textual information can improve the quality and expressiveness of molecular representations.
Shuning Yang, Lei Deng· Journal of Chemical Informat...· 0 citations
Monroe is presented, a new MFM with several innovations over the existing state of the art: increased scale allowing pre-training on over 81 million molecules from the PM6 quantum chemistry dataset; improved graph representation of stereochemistry; improved training losses including conformer denoising and embedding decorrelation; improved multi-task learning; and the use of a prior-data-fitted model (TabPFN) for downstream in-context prediction.
Blazej Banaszewski, Andrew W. Fitzgibbon· 0 citations
Most in silico perturbation methods for single-cell transcriptomics have been validated only on individual datasets, leaving their reliability and generalizability unknown. Through systematic cross-method, cross-dataset benchmarking of eight methods spanning six mathematical frameworks across four datasets, we find that six of eight methods—including widely used VAE-based and tensor decomposition approaches—fail to produce detectable transcription factor (TF)-to-pathway signals. Only CellOracle and DDIM consistently detected TF-to-glycolysis directional regulation. Cross-pathway analysis in PBMC monocytes revealed biologically coherent TF-pathway associations beyond glycolysis (SPI1→glycolysis 4.4× enrichment, FOS→AP-1 targets 4.4×), with SOX9 serving as a biological specificity control (no pathway enrichment). Method choice alone could reverse biological conclusions: DDIM and scTenifoldKnk rankings were significantly anti-correlated (ρ=-0.811, p=0.027). CRISPRi Perturb-seq validation in K562 cells confirmed TF knockdown suppresses glycolysis gene expression (JUN δ=-1.72, CEBPB δ=-1.59, SPI1 δ=-1.57, FOS δ=-0.70), but CellOracle-predicted perturbation directions did not match experimental directions (40.9% agreement, not different from chance), revealing a fundamental gap between steady-state correlation and causal perturbation. Diagnostic analyses using VAE latent space profiling, correlation distribution comparison, and gene-gene graph analysis identified distinct failure modes in unsuccessful methods: VAE latent space competition (STAT3 signal-to-noise 0.44 vs. SPI1 4.25), correlation noise (TF-glycolysis |r|=0.038 indistinguishable from background |r|=0.047), and graph non-specificity (0.84× enrichment). A controlled ablation experiment showed that adding a GRN prior to DDIM did not improve target recall (delta=0 for all TFs), confirming that performance differences are multi-factorial. These findings establish preliminary guidance for method selection, including cross-pathway validation, direction-aware benchmarking, and minimum data requirements (≥500 cells, ≥1,000 HVGs). Author Summary Computational methods that simulate gene knockout experiments from single-cell RNA sequencing data are increasingly popular, but researchers lack guidance on which method to choose. We systematically tested eight such methods across four different cell types, including macrophages from osteoarthritis and rheumatoid arthritis patients, blood monocytes, and leukemia cells. We found that only two methods—CellOracle and DDIM—reliably detected how transcription factors control metabolism. These two methods also detected biologically coherent signals across multiple pathways, not just metabolism. Worryingly, two different methods applied to the same data could produce opposite conclusions about which genes regulate which pathways. We also observed that detecting a perturbation signal does not guarantee predicting its direction correctly: CRISPR-based experimental validation showed that computational methods captured which genes respond to TF perturbation but not whether they are upregulated or downregulated. Through systematic diagnostic analysis, we identified why unsuccessful methods failed: the key TF signals are too weak relative to background variation for purely data-driven approaches to detect. Based on our results, we recommend CellOracle for initial screening (requiring at least 500 cells and 1,000 highly variable genes), cross-pathway validation for any TF→target inference, and orthogonal experimental validation when perturbation direction matters. Our evaluation framework and practical guidelines help researchers choose perturbation methods appropriate for their specific biological questions.
Shixiang Wu, Gang Hu, Zhangquan Yang et al.· bioRxiv· 0 citations
Findings show that performance estimates in protein function prediction should be interpreted as outcomes of complete data-centric workflows rather than isolated properties of predictive models.
