CRUX (CRoss-modal Unification via teXt anchoring), a fusion framework that uses clinical text---encoded by a biomedical language model---as an institution-invariant semantic anchor for robust, generalizable multimodal cancer prognosis, is presented.
Abstract
Accurate prognosis in non-small cell lung cancer (NSCLC) benefits from integrating computed tomography (CT) and histopathological whole-slide images (WSIs), which capture complementary macroscopic and microscopic disease characteristics. Fusing these modalities is difficult: their features come from different pretrained foundation-model encoders and lack spatial correspondence, and both are sensitive to institution-specific acquisition and staining, causing multimodal models to overfit site-specific artifacts and generalize poorly. We present CRUX (CRoss-modal Unification via teXt anchoring), a fusion framework that uses clinical text---encoded by a biomedical language model---as an institution-invariant semantic anchor. Because clinical variables follow standardized classification systems, their embeddings occupy a representation space that is consistent across institutions, providing a stable reference frame for aligning heterogeneous imaging features. CRUX introduces a text-mediated cross-modal bridge in which CT- and pathology-derived text-query summaries exchange information within a compact text semantic space through a learnable hub token, avoiding direct image-space cross-attention that would propagate domain-specific artifacts. We evaluate on 2,157 NSCLC patients from five hospitals, holding out one hospital (N=446) entirely for external validation. CRUX attains an external concordance index (C-index) of 0.761, the highest among all unimodal, bimodal, and trimodal configurations. Notably, direct CT-WSI fusion degrades under distribution shift, whereas text anchoring restores and improves cross-institutional generalization. These results indicate that language-model-encoded clinical text is an effective structured prior for robust, generalizable multimodal cancer prognosis.
PURPOSE
Pretreatment estimation of pathological complete response (pCR) to neoadjuvant chemotherapy may support risk stratification and future response-adapted clinical studies in breast cancer. However, existing deep learning approaches often rely on "black-box" visual pattern recognition, suffering from poor interpre...
Background: Five-year survival prediction from hematoxylin-and-eosin whole-slide images could support risk stratification, but the task is weakly supervised, and the tissue morphologies carrying prognostic information remain incompletely characterized. Methods: We conducted a retrospective secondary analysis using TCGA...
Hikmat Khan, M. Gurcan, A. Parwani et al.· 0 citations
The DLRTransformer improved postoperative risk stratification compared with guideline-based risk stratification and identified patients with stage I NSCLC who benefited from adjuvant therapy.
Li-Ying Wang, Yu Zong, Xin-Yu Li et al.· Radiology: Imaging Cancer· 0 citations
Breast cancer, the most prevalent malignancy among women globally, requires precise assessment of lymph node metastases in whole-slide images (WSIs) for accurate clinical staging and therapeutic planning. However, manual assessment is laborious and prone to inter-observer variability. Inspired by biomimetic principles...
Li-Juan Ren, Hang Chen, Jun Yang et al.· Biomimetics· 0 citations
An explainable AI framework to segment three tissue compartments from H&E patches and constructed 11 spatial pathological features effectively estimate residual cancer burden, suggesting that tumor-rich and lymphocyte-infiltrated necrotic zones correlate with opposing immune phenotypes, supporting personalized breast c...
Xin Shu, Fan Wang, Tian-Cheng Zhao et al.· Diagnostics· 0 citations