Using multimodal foundational models to predict neoantigen immunogenicity and vaccine effectiveness across different tumor types

Abstract

Neoantigen vaccines are a key component of personalised cancer immunotherapy; however, existing prediction techniques are primarily restricted to a single tumour type and have difficulty integrating multi-modal biological data, which leads to inadequate accuracy in immunogenicity evaluation and vaccine efficacy prediction. In order to predict neoantigen immunogenicity and customised vaccination clinical outcomes across various tumour types, this study attempts to develop and verify a universal multi-modal foundation model.Neoantigen peptide sequences, mass spectrometry-derived pHLA binding patterns, single-cell TCR repertoires, tumour transcriptomes, and clinical vaccination trial follow-up records were among the extensive paired data we gathered from 15 solid tumour types. We present the NeoVAX-FM multi-modal foundation model, which first embeds peptide sequences, pHLA complex 3D structures, and gene expression into a single semantic space using a contrastive language-image pretraining paradigm. It is then fine-tuned on downstream tasks to concurrently score immunogenicity and predict progression-free survival.The model was assessed in one prospective clinical trial and three external validation cohorts. NeoVAX-FM showed strong performance in melanoma, non-small cell lung cancer, and microsatellite stable colorectal cancer, with an average AUC of 0.94 for cross-tumor neoantigen immunogenicity prediction—a 12.3% increase over the best currently available techniques. Patients in the prospective vaccination cohort who were projected by the model to be “high responders” had a considerably higher median progression-free survival (HR = 0.28, p < 0.001), and the model was successful in identifying tumour microenvironment characteristics and universal TCR motifs that drive long-term responses.This work is the first to use a multi-modal foundation model for neoantigen vaccines, overcoming the tumor-type-specific constraints of conventional approaches and facilitating joint modelling of immunogenicity and clinical efficacy, thereby offering an AI decision engine for precision cancer vaccine design that is applicable to all cancer types.