Genome-scale molecular profiling from routine histopathology with a multimodal pathology foundation model
Pathology foundation models trained from whole-slide images alone treat tissue morphology as an autonomous visual phenotype, leaving learned representations only weakly anchored to the molecular processes that generate tissue architecture. Here we present Fuji, a multimodal pathology foundation model that grounds morphology in transcriptomic and mutational supervision through a two-stage pretraining strategy: multi-teacher contrastive distillation that harmonizes formalin-fixed paraffin-embedded and fresh-frozen specimens, followed by multimodal multi-task learning that jointly optimizes masked image reconstruction, transcriptomic embedding reconstruction, and attention-aligned mutational signature inference. Trained on 60,546 whole-slide images paired with 45,675 bulk RNA-seq profiles and 22,683 whole-genome sequencing samples drawn from TCGA, GTEx, and CPTAC, Fuji yields a protocol-invariant, genome-informed embedding that we treat as a quantitative coordinate system for probing how genomic states project into tissue architecture. Genome-scale instability phenotypes, microsatellite instability, chromosomal instability, homologous recombination deficiency, and whole-genome duplication, are recoverable from histology with AUCs up to 0.98, consistently exceeding existing pathology foundation models. Extrachromosomal DNA amplification leaves a distinct morphological footprint that stratifies survival independently of tissue lineage. Etiologic mutational processes such as tobacco exposure and mismatch repair deficiency localize to discrete architectural neighborhoods, validated in two independent population-based cohorts, whereas clock-like signatures remain diffuse, delineating the scope and limits of genomic visibility from routine histology. Fuji further enables direct inference of tumor mutational burden, neoantigen load, immunotherapy response, and FDA-actionable driver mutations from hematoxylin and eosin slides. By coupling pretraining to molecular and mutational supervision, Fuji establishes an engineering basis for genome-informed inference from the most widely performed assay in oncology.