Skip to content

CRUX: Clinical-Text Anchoring for Cross-Institutional CT-Pathology Fusion in Non-Small Cell Lung Cancer Survival Prediction

Oct 2026 · 0 citations · 26 references
Engineering

TL;DR

CRUX (CRoss-modal Unification via teXt anchoring), a fusion framework that uses clinical text---encoded by a biomedical language model---as an institution-invariant semantic anchor for robust, generalizable multimodal cancer prognosis, is presented.

Abstract

Accurate prognosis in non-small cell lung cancer (NSCLC) benefits from integrating computed tomography (CT) and histopathological whole-slide images (WSIs), which capture complementary macroscopic and microscopic disease characteristics. Fusing these modalities is difficult: their features come from different pretrained foundation-model encoders and lack spatial correspondence, and both are sensitive to institution-specific acquisition and staining, causing multimodal models to overfit site-specific artifacts and generalize poorly. We present CRUX (CRoss-modal Unification via teXt anchoring), a fusion framework that uses clinical text---encoded by a biomedical language model---as an institution-invariant semantic anchor. Because clinical variables follow standardized classification systems, their embeddings occupy a representation space that is consistent across institutions, providing a stable reference frame for aligning heterogeneous imaging features. CRUX introduces a text-mediated cross-modal bridge in which CT- and pathology-derived text-query summaries exchange information within a compact text semantic space through a learnable hub token, avoiding direct image-space cross-attention that would propagate domain-specific artifacts. We evaluate on 2,157 NSCLC patients from five hospitals, holding out one hospital (N=446) entirely for external validation. CRUX attains an external concordance index (C-index) of 0.761, the highest among all unimodal, bimodal, and trimodal configurations. Notably, direct CT-WSI fusion degrades under distribution shift, whereas text anchoring restores and improves cross-institutional generalization. These results indicate that language-model-encoded clinical text is an effective structured prior for robust, generalizable multimodal cancer prognosis.

View source

Similar papers

Open access Oct 2026

Knowledge-guided Multimodal Learning for Interpretable Prediction of Pathological Complete Response and Survival in Breast Cancer.

PURPOSE Pretreatment estimation of pathological complete response (pCR) to neoadjuvant chemotherapy may support risk stratification and future response-adapted clinical studies in breast cancer. However, existing deep learning approaches often rely on "black-box" visual pattern recognition, suffering from poor interpre...

Wen-Chuan Zhang, Shu-Wan Zhang, Ya-Ni Wei et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Morphology-aware deep learning predicts colorectal cancer survival from whole-slide images

Background: Five-year survival prediction from hematoxylin-and-eosin whole-slide images could support risk stratification, but the task is weakly supervised, and the tissue morphologies carrying prognostic information remain incompletely characterized. Methods: We conducted a retrospective secondary analysis using TCGA...

Hikmat Khan, M. Gurcan, A. Parwani et al. · 0 citations
#small language model Open access Nov 2026

Multimodal Transformer Integrating CT and Pathology for Postoperative Risk Stratification in Stage I Non-Small Cell Lung Cancer.

The DLRTransformer improved postoperative risk stratification compared with guideline-based risk stratification and identified patients with stage I NSCLC who benefited from adjuvant therapy.

Li-Ying Wang, Yu Zong, Xin-Yu Li et al. · 0 citations
Open access Oct 2026

HT-Net: A Task-Specific Dual-Branch Network for Tumor Segmentation in Histopathological Images

Breast cancer, the most prevalent malignancy among women globally, requires precise assessment of lymph node metastases in whole-slide images (WSIs) for accurate clinical staging and therapeutic planning. However, manual assessment is laborious and prone to inter-observer variability. Inspired by biomimetic principles...

Li-Juan Ren, Hang Chen, Jun Yang et al. · 0 citations
Open access Aug 2026

Explainable AI-Derived Spatial Pathological Features of Tumor, Necrosis, and Lymphocytes Identify Key Histological Signatures for Residual Cancer Burden Assessment in Breast Cancer

An explainable AI framework to segment three tissue compartments from H&E patches and constructed 11 spatial pathological features effectively estimate residual cancer burden, suggesting that tumor-rich and lymphocyte-infiltrated necrotic zones correlate with opposing immune phenotypes, supporting personalized breast c...

Xin Shu, Fan Wang, Tian-Cheng Zhao et al. · 0 citations

Related blog posts

Google DeepMind Blog Nov 18, 2026

Genie 4 is now available

We are rolling out a new Project Genie experience to eligible Google AI Ultra subscribers, opening access to our new Genie 4 world model.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.