DeMoP: A Language-Model-Guided Mixture-of-Experts Framework for Cancer Prognosis

Abstract

Integrating heterogeneous clinical and molecular data for cancer prognosis remains challenging because their dimensionality, semantics and distributions differ across patients and cohorts. Here we present DeMoP, a language-model-guided mixture-of-experts framework that serializes structured patient profiles as natural-language sequences and learns adaptive prognostic representations from clinical variables, copy-number alterations, and gene descriptions. DeMoP combines a fine-tuned DeBERTa-v3-large encoder, attention-based token pooling, and a residual mixture-of-experts prediction head. In held-out tests from two independent pan-cancer cohorts, GENIE (63,090 patients) and TCGA (4,123 patients), DeMoP outperformed the conventional machine-learning and deep-learning baselines evaluated, achieving AUROCs of 0.939 and 0.805 and class-1 F1 scores of 0.72 in both cohorts. A GENIE-trained model transferred directly to TCGA with an overall class-1 F1 score of 0.62. Gene-level ablations recovered established cancer-associated genes and highlighted less-studied candidates. DeMoP provides a unified approach to heterogeneous biomedical data integration, cross-cohort outcome prediction, and model interpretation.