DFCE-KanT: predicting spatial gene expression from histology images via contrastive learning
Abstract Motivation Spatial transcriptomics (ST) can quantitatively characterize the spatial molecular profile in tissues and yet is costly to be applied at a large scale. One viable, a low-cost alternative would be to directly predict spatial gene expression from routine hematoxylin eosin (H&E) stained WSIs. Yet, the spatial context of the H&E image corresponding to a ST is not effectively utilized in current deep learning approaches. Results We present DFCE-KanT, which is a contrastive learning framework combining the tissue image features, gene features, and spatial location information for predicting spatial gene expressions from HE images. DenseNet incorporated a Feature Channel Enhancement (FCE) attention technique to extract features from the H&E images. A KanT, which is composed of a Kolmogorov-Arnold Network (KAN) and Transformer multi-head attention that can learn the best fusion strategy automatically in terms of fusing space information via positional encoding, increasing the ability to capture complex data patterns. This structure enhances the nonlinear ability for the projection head, modeling complex interaction between features and promoting the compatibility between image and gene expression features in the common embedding space. Experiments on five publicly available datasets (HER2+, cSCC, Alex, HBC, and Liver) demonstrate that DFCE-KanT performs noticeably better than existing methods, validating its efficacy in spatial gene expression prediction. Availability The source code and data are available at GitHub (https://github.com/LFfocus/DFCE-KanT) and Zenodo (https://zenodo.org/records/20637079).