Skip to content

M-JEPA: Predictive Self-Supervised Learning for Molecular Graphs with Scaffold-Shift Evaluation on Tox21.

Jul 2026 · Journal of Chemical Information and Modeling · Vol 66, pp. 8008-8021 · 1 citation · 7 references
Medicine

TL;DR

M-JEPA (Molecular Joint Embedding Predictive Architectures), a predictive self-supervised method for molecular graphs based on connected-subgraph masking and an exponential-moving-average (EMA) teacher, is evaluated with a compute-matched three-phase protocol that screens objectives on ESOL and tests transfer on Tox21 under Bemis-Murcko scaffold splits.

Abstract

Self-supervised molecular representation learning can improve transfer on label-limited property prediction tasks, but contrastive objectives are sensitive to view construction and are often evaluated using protocols that report discrimination alone. We introduce M-JEPA (Molecular Joint Embedding Predictive Architectures), a predictive self-supervised method for molecular graphs based on connected-subgraph masking and an exponential-moving-average (EMA) teacher, and evaluate it with a compute-matched three-phase protocol that screens objectives on ESOL and tests transfer on Tox21 under Bemis-Murcko scaffold splits. Under matched compute, M-JEPA achieves lower ESOL proxy RMSE than a compute-matched InfoNCE baseline (2.12 vs 3.75; paired ΔRMSE 1.61, 95% CI 1.52-1.72; Wilcoxon p = 2.4 × 10-4) and shorter Phase-1 wall-clock time under the present implementation (24.09 ± 2.53 vs 44.21 ± 4.03 min on a single GPU). On Tox21, hybrid fine-tuning from the M-JEPA checkpoint improves mean ROC-AUC from 0.561 to 0.609 and reduces mean ECE from 0.279 to 0.064 relative to a supervised-from-scratch baseline under the same scaffold split, with paired cross-assay tests supporting systematic rather than assay-specific benefits (Wilcoxon p ≤ 5 × 10-3 for all four metrics). Motif-level Integrated Gradients attributions are stable across fine-tuning in most cases (median Spearman ρ = 0.90 across 62 molecule-assay pairs), although attribution stability and discrimination gains are end point-dependent. Scope. The conclusions in this study are specific to Tox21 scaffold-shift transfer with predictive versus contrastive self-supervision under matched compute; generalization to other MoleculeNet end points is a natural next step and is left for future work.

View source

Similar papers

#machine learning Preprint Aug 2026

ToxLens: A Reproducible Graph-Learning Framework for Leakage-Aware, Uncertainty-Calibrated Molecular Toxicity Prediction

Molecular toxicity prediction is increasingly used to prioritise compounds before experimental testing, but conventional benchmark performance can overstate practical utility when structurally related molecules occur across training and test folds. We introduce ToxLens, a reproducible multi-task graph-learning framework for 11 toxicity endpoints spanning Ames mutagenicity, acute oral toxicity, hERG inhibition, and Tox21 nuclear-receptor and stress-response assays. The workflow combines conservative chemical curation, sphere-exclusion filtering, a leakage-aware UMAP-HDBSCAN split, parallel graph and global-feature encoders joined by late concatenation, temperature-scaled Monte Carlo dropout with conformal-style prediction sets, applicability-domain analysis, and SHAP-guided toxicophore discovery with occlusion controls. On the leakage-controlled test fold, a five-seed soft-voting ensemble achieved a Matthews correlation coefficient score of 0.44, an area under the receiver operating characteristic curve score of 0.83, and an area under the precision-recall curve score of 0.58. It exceeded four ECFP4-based shallow baselines on all 11 endpoints under the same split and validation-based threshold-selection protocol. Controlled ablations showed that the global pathway was important, whereas late concatenation outperformed the tested gated and feature-wise linear modulation fusion variants. Conformal-style prediction sets revealed substantial endpoint-specific variation in set efficiency, and discrimination and calibration improved with similarity to the training domain. Retraining on fixed published Tox21 Challenge and TDA folds produced competitive, but not uniformly state-of-the-art, performance. SHAP-guided occlusion and consensus subgraph mining yielded model-derived structural hypotheses, 44 of which contained at least one occurrence that passed the predefined counterfactual criteria.

Magnus H. Strømme, A. D. de Sá, David B. Ascher · 0 citations
Preprint Aug 2026

bioMoR: Biology-Guided Mixture-of-Recursions for Effective Genomic Learning

This work proposes bioMoR, which is the first framework to apply MoR to gene-level and pathway-level learning, and identifies three locations for integrating structured biological knowledge within an MoR backbone: graph-based information sharing refines token embeddings, a structural bias guides self-attention toward biologically related tokens, and a graph-aware router uses neighborhood information to determine each token's recursion depth.

Koushik Howlader, Tirtho Roy, Md Tauhidul Islam et al. · 0 citations
Preprint Jul 2026

Improving Molecular Property Prediction in Small Language Models Using Graph-based Tools

A modular Context-Augmented Prompting framework that enables agentic tool use at inference time: a trained GNN expert model provides a predictive hint with confidence, and a GNN extracts an instance-specific explanatory subgraph via a necessity-based edge-drop intervention.

K. Bougiatiotis, Dimitrios Kelesis, Georgios Paliouras · 1 citation
Preprint Aug 2026

BioM-JEPA: joint-embedding prediction of graph-connected gene blocks in single cells

Results support graph-connected gene blocks as useful prediction units for JEPA-style representation learning in single-cell biology by supporting block-level prediction of graph-connected gene blocks defined by protein-association and corpus-derived coexpression evidence.

Yuhao Wang, Zelin Zang, Yuxuan Liu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.