The oFM is introduced, a foundation model developed on a real-world oncology cohort of 1.67 million cancer patients that integrates clinical trajectories with DNA, RNA, and H&E pathology and achieves a three-fold higher pooled and scale-normalized treatment-benefit AUTOC than baseline features.
Abstract
Precision oncology necessitates a longitudinal model of patient state that captures cancer evolution and treatment over time, integrating multimodal observations. We introduce the oFM, a foundation model developed on a real-world oncology cohort of 1.67 million cancer patients that integrates clinical trajectories with DNA, RNA, and H&E pathology. Patient-level partitions were reserved for training, validation, and testing, with over one million patients used for training. The oFM encodes daily clinical and molecular episodes and, along with pathology images, integrates them over time to produce a patient state embedding. We evaluate frozen oFM embeddings against expert-curated clinical and molecular baseline features. In prognostic benchmarks, the oFM improved AUC for treatment response, progression-free survival, and overall survival (0.774 vs. 0.563 for overall survival). Across 11 comparative-treatment cohorts, the oFM embeddings achieved a three-fold higher pooled and scale-normalized treatment-benefit AUTOC than baseline features with improved benefit ranking in 9 of 11 cohorts, and provided stronger prognostic discrimination within both treatment arms. We also evaluated a mechanism discovery framework that interprets downstream models built on oFM embeddings by linking their predicted outcomes to clinically and biologically grounded mechanisms through an evidence-grounded temporal graph, enabling evaluation in clinical and drug-development applications.
Effective glioblastoma care requires integrating multiparametric longitudinal MRI with histopathology, molecular profiling, and clinical records documenting surgery, radiotherapy, chemotherapy, and supportive treatments. In routine practice, however, these data streams are often evaluated separately rather than jointly, which can delay molecularly informed stratification, limit reproducibility across centers, and complicate interpretation of post-treatment imaging changes. Multimodal machine learning (MML) provides a framework for clinical decision support by integrating diverse patient data across the course of care, from symptom presentation through diagnosis to treatment decisions. By combining MRI, whole-slide pathology, molecular and methylation profiling, and treatment timelines derived from electronic health records, MML models can capture disease characteristics over time across biological scales through representation learning and multimodal fusion. Importantly, these approaches can incorporate uncertainty through model calibration and confidence-aware predictions. When rigorously developed and validated, MML models may generate clinically relevant outputs, including integrated diagnosis, molecular classification, individualized survival estimates, probabilistic discrimination between tumor progression and pseudo-progression, and stratification for clinical trial eligibility. In this Mini Review, we summarize recent advances and emerging translational evidence for clinically oriented MML in glioblastoma, with particular emphasis on MRI-centered systems that integrate imaging with pathology, selected molecular measurements, and longitudinal clinical context. We also outline key methodological and practical considerations, including dataset curation, leakage control, external validation, calibration, and post-deployment monitoring—required to support safe, robust, and generalizable implementation of MML approaches in neuro-oncology practice.
Amin Zadeh-Shirazi, Bryan W. Day, Hui K. Gan et al.· Frontiers in Oncology· 0 citations
A multi-tier, explainable AI framework designed to risk-stratify patients and predict overall survival using clinical and genomic covariates is developed and demonstrates that explainable machine learning models can robustly predict survivability and highlight actionable features for oncology dashboards.
Accurate prediction of tumor recurrence in brain tumor patients following surgery is essential for optimizing adjuvant therapy, response assessment, and surveillance regimen. While MRI remains the gold standard for surveillance, integrating patient-specific clinical context may inform recurrence prediction. Traditional multimodal deep learning approaches often incorporate clinical data via simple fusion, failing to fully capture the semantic interdependencies between visual features and clinical context. Trained on over 5,000 scans from approximately 400 pediatric low-grade glioma subjects and validated across three institutional cohorts, including one clinical trial cohort, our experiments demonstrate incremental performance gains when progressing from vision-only to clinical-vision to a vision-language approach. Our results indicate that converting structured clinical covariates into natural language text allows for more effective synthesis of multimodal data, while providing a platform for incremental addition of clinical context without extending model complexity. We demonstrate that our proposed VLM architecture offers a promising direction for neuro-oncological prognosis by effectively encoding imaging cues and clinical context, with potential applicability to other longitudinal prognosis tasks.
D. Tak, D. Sreedhar, H. Aerts et al.· medRxiv· 0 citations
MRI foundation models (FMs) have shown potential for improving performance of neuroimaging tasks, but their value specifically in Glioblastoma overall survival risk prediction remains unclear. In this study, we explored foundation model initialization for preoperative risk prediction using publicly available structural MRIs from 1007 patients across three glioblastoma cohorts: UPENN-GBM, UCSF-PDGM and TCGA-GBM. The primary analysis finetuned a Swin vision transformer, initialized with BrainSegFounder weights, and compared performance to matched random initialization, and a radiomics baseline. We additionally evaluated recently released FMs, including BrainMVP, BrainIAC, and TRIAD, within the same adaptation strategy, and explored multimodal integration of diffusion tensor imaging-based risk predictions, age, extent of surgical resection, and MGMT methylation status into predictions. In 10-fold UPENN-GBM cross-validation, BrainSegFounder improved mean C-index compared to matched training from scratch and radiomics (0.667 ± 0.053 vs 0.649 ± 0.040 and 0.612 ± 0.044) and achieved time-dependent AUROCs between 0.766 ± 0.089 and 0.799 ± 0.073 across survival horizons under one year. All tested FMs improved over matched random initialization. Within complete-case subsets, multimodal integration improved performance (C-Index 0.691 ± 0.053). Leave-one-cohort-out validation showed performance in external settings consistent with the broader GBM survival prediction literature. These findings suggest incremental value of FM initialization for GBM risk stratification and supports continued benchmarking, adaptation, and validation.
Rakesh Khanna, Fanyang Yu, Minkyu Park et al.· npj Precision Oncology· 0 citations
Cancer management remains fragmented across its continuum, from late-stage diagnosis and salvage therapies to non-personalized surveillance. Here, we present Oncoformer, a unified multimodal transformer model trained on the China Oncology Multimodal Prediction and Surveillance Study (COMPASS) cohort (3.67 million individuals, 17.7 million clinical visits) and validated on independent external cohorts, including the UK Biobank. Oncoformer integrates longitudinal electronic health records with chest X-ray imaging to address multiple clinical tasks: pan-cancer diagnosis (area under the receiver operating characteristic curve [AUROC] = 0.956), future cancer prediction up to 1 year before diagnosis (AUROC = 0.869), tumor stage inference (mean AUROC > 0.90), patient-specific treatment-response forecasting, and recurrence-free survival stratification across ten cancer types (all p < 0.01). Staging predictions were independently validated against postoperative pathological endpoints and shown to converge on core cancer genomic pathways. By translating routine clinical data into a dynamic view of cancer evolution, Oncoformer provides a framework for risk-informed cancer prediction and treatment stratification using routine clinical data.
Fei Liu, Kai Wang, Hui Xu et al.· Cell· 1 citation
This review synthesizes key developments in ML for oncology, covering foundational algorithms alongside emerging approaches, and describes future directions, including federated learning, graph neural networks, longitudinal modeling, and integration of real-world and wearable data to support precision oncology.
Kanishk Yadav, Taneesha Gupta· Journal of the Egyptian Nati...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
