An Explanatory Deep Learning Pipeline for the Prediction and Visualization of Spatially Resolved Biomarker Expression in Triple-Negative Breast Cancer.
Aug 2026· American Journal of Pathology· 0 citations· 38 references
Medicine
TL;DR
X-SPATIO is a spatially compatible computational pipeline designed to directly link hematoxylin and eosin morphology with region-matched mRNA and protein expression, enabling cost-effective inference of spatial biomarker expression and establishing a foundation for biologically grounded discovery and precision oncology in TNBC.
Abstract
Histopathologic evaluation remains central to cancer diagnosis and treatment planning, yet the molecular programs underlying distinct tissue morphologies aren't routinely accessible in clinical workflows. Spatial transcriptomic/proteomic platforms provide region-specific molecular measurements but are limited by cost, throughput, and scalability. Most computational pathology models rely on either bulk tissue-based gene expression or a focused gene/protein expression-panel prediction, thereby obscuring subregion-specific morpho-molecular relationships and limiting spatial interpretation of a wider gene/protein expression network. This limitation is particularly significant in triple-negative breast cancer (TNBC), which exhibits pronounced spatial heterogeneity across tumor, stroma, and immune compartments. We developed X-SPATIO, a spatially compatible computational pipeline designed to directly link hematoxylin and eosin (H&E) morphology with region-matched mRNA and protein expression. The model was trained on H&E-defined regions of interest paired with spatially-resolved omics data obtained from GeoMx Digital Spatial Profiling. Using a multiple-instance learning approach, X-SPATIO captures morpho-molecular associations, generating spatio-morphologic attention maps that indicate predictive tissue regions. X-SPATIO demonstrated strong performance across biologically relevant spatial biomarkers, achieving area under the curve values ranging from 0.79-0.97. Attention maps revealed spatial patterns consistent with known biology, indicating alignment between learned features and tissue organization. By integrating spatial molecular ground truth with routine histopathology, X-SPATIO enables cost-effective inference of spatial biomarker expression and establishes a foundation for biologically grounded discovery and precision oncology in TNBC.
Abstract Motivation Spatial transcriptomics enables spatially-resolved measurement of gene expression in tissues, but its widespread adoption has been limited by high cost. Predicting gene expression profiles from histology images using deep learning has recently attracted broad interest as a promising and cost-effective spatial transcriptomics solution, yet the strengths and limitations of existing methods have not been comprehensively assessed. Results We reviewed current approaches and compared seven algorithms across six datasets spanning four cancer types and a non-cancer disease, covering three spatial protocols and a comprehensive set of performance measures. Unlike previous studies that focused only on highly variable genes, we also analysed the predictability of genes functionally relevant to disease. We further assessed out-of-domain performance, the effect of data transformation techniques, the use of predicted gene expression for detecting tissue domains, and the benefit of integrating histopathology foundation models. Our results provide new quantitative insights into the limitations and promise of deep learning based spatial transcriptomics methods, and suggest challenges and future directions. Availability and implementation The benchmarking pipeline and source code are available at https://github.com/BiomedicalMachineLearning/DeepHis2Exp.
Spatial transcriptomics reveals cellular heterogeneity, intercellular communication, and tissue organization, but its cost and limited accessibility restrict clinical use. Here, we present VISTA, a model that integrates multi-scale histological features and spatial context to infer spatial gene expression from H&E-stained tissue images. Across leave-one-section-out cross-validation and independent validation, VISTA robustly predicted thousands of genes and outperformed state-of-the-art methods. Beyond expression reconstruction, VISTA enabled clinically relevant downstream analyses. In TCGA breast cancer samples, it identified survival-associated genes, stratified prognostic risk groups, and revealed adverse tumor-associated spatial subtypes. In our in-house intrahepatic cholangiocarcinoma cohort, it preserved tumor–normal organization and identified CLDN4 and CYP3A4 as complementary spatial biomarkers. In HER2+ breast cancer, it predicted pathological response to neoadjuvant trastuzumab-based therapy and linked response-associated regions to immune and cytokine-related programs. These results support virtual spatial transcriptomics from routine histopathology for oncology applications.
Shaoqing Jiao, Zhen Yuan, Dazhi Lu et al.· bioRxiv· 0 citations
VirTues-derived biomarkers predict anti-PD-L1 chemo-immunotherapy response and stratify disease-free survival in an independent cohort, outperforming state-of-the-art biomarkers derived from the same datasets and current clinical stratification schemes.
Johann Wenckstern, Eeshaan Jain, Benedikt von Querfurth et al.· Nature· 1 citation
Spatial transcriptomic analyses provide spatially resolved gene expression data that can provide insights into complex biological processes. However, current spatial transcriptomics approaches remain financially prohibitive and restricted in resolution, scalability, and gene coverage, limiting broader adoption for large-scale studies. Here, we developed CarHE (contrastive alignment of gene expression for hematoxylin and eosin images), a multimodal pretraining framework that infers high-dimensional spatial transcriptomic profiles from routine H&E-stained slides. By using contrastive learning to align cell type-specific transcriptomic information with histological features, CarHE achieved high prediction accuracy across evaluated datasets and spatial transcriptomics platforms. CarHE approximated spatially organized pathological microenvironment features consistent with tertiary lymphoid structure (TLS)-associated regions in breast cancer, lung cancer, melanoma, and clear cell renal cell carcinoma. Additionally, CarHE inferred approximated 3D spatial transcriptomic context from 2D images, providing more informative neighborhood context than 2D visualization. In a cohort of 880 lung cancer patients, CarHE-derived features were associated with disease-free survival and outperformed current approaches. Overall, CarHE provides a cost-effective and scalable framework for H&E-based spatial inference, supporting further validation toward translational research applications.
Jiawei Zou, Kai Xiao, Zexi Chen et al.· Cancer Research· 0 citations
Spatial omics (SO) technologies enable spatially resolved molecular profiling, while hematoxylin and eosin (H&E) imaging remains the gold standard for morphological assessment in clinical pathology. Recent computational advances increasingly place H&E images at the center of SO analysis, bridging morphology with transcriptomic, proteomic, and other spatial molecular modalities. This lecture-style tutorial surveys the algorithmic foundations and recent advances in method development that make them practical and impactful for precision medicine. Following the tutorial flow, we first introduce key SO modalities and data abstractions (tiles/patches, spots, cells, and spatial graphs) and articulate problems to address and motivations, emphasizing multi-scale mismatch, structured spatial dependence, weak supervision, and domain shift across cohorts and sites. We then trace the evolution of modern multimodal representation learning, highlighting graph neural networks, transformer-based architectures, and encoder–decoder designs. The core of the tutorial systematically organizes contemporary methods into three categories: (i) integration methods, which jointly model paired multimodal measurements; (ii) mapping methods, which predict spatial molecular profiles from H&E images; and (iii) foundation models (FMs), which learn transferable representations from large-scale spatial datasets via self-supervised pretraining, contrastive objectives, etc. This tutorial also discusses applications of generative modeling to support imputation and data augmentation. Throughout, we connect methods to real biomedical endpoints (e.g., tumor microenvironment characterization, biomarker discovery, and cohort-level stratification). We further summarize actionable modeling directions enabled by current architectures and delineate persistent gaps driven by data, biology, and technology that are unlikely to be resolved by model design alone. The tutorial concludes with open challenges in interpretability, reliability, privacy, and clinical translation, outlining opportunities for the KDD community to contribute principled data mining and learning approaches to multimodal spatial biology.
Ninghui Hao, Boshen Yan, Dong Li et al.· Proceedings of the 32nd ACM...· 0 citations
Recent advances in spatially resolved transcriptomics have enabled large-scale measurement of gene expression while preserving spatial context, facilitating the investigation of spatial heterogeneity within tissues. In this study, we propose SpatialGEO, a geometric-aware deep learning framework that integrates gene expression profiles with spatial coordinates to generate biologically meaningful low-dimensional embeddings, enabling the dissection of complex tissue architectures. We systematically evaluate SpatialGEO across multiple tissue types and diverse SRT platforms. Results show that SpatialGEO achieves superior performance in tissue structure dissection and data denoising compared to state-of-the-art methods. Moreover, when applied to human breast cancer samples, SpatialGEO precisely delineates the tumor microenvironment and uncovers molecular heterogeneity within tumors and intercellular communication between invasive ductal carcinoma and tumor edge. In mouse embryogenesis, SpatialGEO accurately reconstructs spatiotemporal tissue architectures, highlighting organ-specific developmental programs and elucidating molecular drivers of early neural development.
