Skip to content
Book Open access

CGFusion: Concept-Guided Multimodal Fusion for Interpretable Cancer Diagnosis and Prognosis

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · pp. 12192-12201 · 0 citations · 41 references

TL;DR

CGFusion, an interpretable multimodal framework that jointly models representation alignment and concept-level semantic grounding, is proposed, suggesting that explicitly coupling multimodal alignment with concept-level grounding is a promising direction for building reliable and interpretable multimodal medical systems.

Abstract

Integrating radiology, whole-slide pathology, and structured clinical data is essential for reliable cancer prediction, yet remains challenging due to substantial cross-modal heterogeneity. Existing multimodal models often fail to establish consistent cross-modal representations and provide limited semantic interpretability, making their predictions difficult to trust in clinical settings. To address these challenges, we propose CGFusion, an interpretable multimodal framework that jointly models representation alignment and concept-level semantic grounding. First, a patient-centric contrastive module explicitly aligns CT and WSI representations at the patient level, reducing semantic inconsistency between macroscopic radiology and microscopic pathology. Second, structured clinical variables are projected to initialize the latent queries of a Perceiver-based fusion network, enabling patient-specific priors to guide cross-modal interaction from the outset. Third, an LLM-guided concept grounding module anchors fused latent tokens to clinically meaningful prototypes through temperature-scaled similarity, token-dependent abstention, and hierarchical regularization. Experiments on NLST and MMIST ccRCC demonstrate that CGFusion achieves strong overall performance on NLST and delivers more balanced predictions than competitive baselines on MMIST under severe class imbalance. These results suggest that explicitly coupling multimodal alignment with concept-level grounding is a promising direction for building reliable and interpretable multimodal medical systems.

Read PDF

Similar papers

Open access Oct 2026

Knowledge-guided Multimodal Learning for Interpretable Prediction of Pathological Complete Response and Survival in Breast Cancer.

PURPOSE Pretreatment estimation of pathological complete response (pCR) to neoadjuvant chemotherapy may support risk stratification and future response-adapted clinical studies in breast cancer. However, existing deep learning approaches often rely on "black-box" visual pattern recognition, suffering from poor interpre...

Wen-Chuan Zhang, Shu-Wan Zhang, Ya-Ni Wei et al. · 0 citations
Open access Oct 2026

Anchor-driven Multi-modal Multi-scale Expert Selection for Survival Prediction

The integrative analysis of histopathological Whole-Slide Images (WSIs) and transcriptomic profiles holds significant promise for cancer survival prediction. However, existing methods typically project multi-modal features directly into a shared latent space without explicit alignment, leading to the entanglement of mi...

Tao Zhou, Ying Hu, Hua-Zhu Fu et al. · 0 citations
#small language model Preprint Oct 2026

CRUX: Clinical-Text Anchoring for Cross-Institutional CT-Pathology Fusion in Non-Small Cell Lung Cancer Survival Prediction

CRUX (CRoss-modal Unification via teXt anchoring), a fusion framework that uses clinical text---encoded by a biomedical language model---as an institution-invariant semantic anchor for robust, generalizable multimodal cancer prognosis, is presented.

Kyungwon Kim, Dosik Hwang · 0 citations
Open access Oct 2026

Case-Specific Priors-Guided Multimodal Fusion for Prognosis Prediction in Head and Neck Cancer

Accurate prognosis prediction for head and neck cancer typically necessitates integrating highly heterogeneous clinical and pathological evidence by sophisticated multimodal fusion. However, existing fusion techniques are usually grounded in case-generic clinical rationales about complementary synergies across modaliti...

Yue-Zhao Lin, Shu-Chang Ye, Sheng-Hai Wu et al. · 0 citations
Book Open access Nov 2026

Disentangling Consistency, Complementarity, and Noise for Multimodal Cancer Survival Prediction

Integrating heterogeneous cancer data, such as histopathological images and genomic profiles, provides a multimodal perspective for assessing patient survival and improves predictive performance. However, existing methods often overlook the complex entanglement among multimodal shared prognostic information (consistenc...

Geng Gao, Kai-Xuan Zhang, Ni-Ni Rao · 0 citations
#artificial intelligence Preprint Oct 2026

MedCORE: Criteria-Grounded Clinical Reasoning for Interpretable Medical Image Diagnosis

Clinical diagnosis is inherently a structured reasoning process, yet existing deep learning models often bypass this structure by mapping image features directly to disease labels without explicitly interrogating the morphological and textural criteria that clinicians systematically evaluate. This limits diagnostic tra...

Asim Khan, Samee Ullah Khan, D. Mahapatra · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.