Aug 2026· Applied Sciences· 0 citations· 20 references
TL;DR
Results suggest that PLAT provides an effective and interpretable framework for high-dimensional transcriptomic classification and functional enrichment analyses consistently highlighted biological processes and disease pathways associated with breast cancer, supporting the biological relevance of the learned latent representations.
Abstract
High-dimensional omic datasets present major challenges for machine learning due to their sparse biological signal, strong feature heterogeneity, and high dimensionality. In this work, we propose PLAT (Parallel Latent Attention Transformer), a neural architecture for high-dimensional tabular transcriptomic data. The model projects input gene expression features into multiple parallel latent representations, each processed independently through self-attention to capture complementary feature interactions while maintaining moderate model complexity. The proposed architecture was evaluated using both controlled Negative Binomial simulations designed to reproduce RNA-seq overdispersion and the TCGA-BRCA breast cancer dataset comprising 499 patients and 4376 gene expression variables for ER+/ER− classification. Comparative analyses against a baseline multilayer perceptron and a lightweight FT-Transformer showed that PLAT achieves competitive predictive performance while maintaining a comparable number of trainable parameters. Simulation experiments further indicate that its main advantage is concentrated in specific high-dimensional settings with an intermediate proportion of informative features. To assess model interpretability, we additionally performed a SHAP-based analysis of the baseline MLP and compared it with the attention-derived gene rankings. Although both models identified largely different sets of predictive genes, functional enrichment analyses consistently highlighted biological processes and disease pathways associated with breast cancer, supporting the biological relevance of the learned latent representations. These results suggest that PLAT provides an effective and interpretable framework for high-dimensional transcriptomic classification.
In biomarker discovery, access to sufficient quantities of condition-specific transcriptomic data is often limited by cohort size, privacy concerns, and domain shift between normal and condition populations. Generative modeling can augment scarce cohorts and probe distributional transitions. Furthermore, synthetic transcriptome generation can support differential expression analyses, machine learning, privacy-preserving data sharing, benchmarking, and hypothesis generation in translational bioinformatics workloads in fields such as oncology. Here, we present Hoike, a framework that combines a crossdomain Joint-Embedding Predictive Architecture (JEPA) with a latent diffusion model to generate condition-specific bulk transcriptomes from a normal reference context. In Hoike, normal tissue profiles provide continuous conditioning signals, while the model learns disease-linked shifts in latent space and reconstructs gene-level expression in log2(TPM+1) space. The implementation supports paired normal-condition training, tissuealigned conditioning, and constrained non-negative decoding for biologically valid outputs. We describe the architecture, objective design, and evaluation protocol used in this work across GTEx-derived normal references and multiple TCGA condition cohorts as a case study. This serves as the technical specification of the Hoike framework and its reproducible analysis workflow.
Omics datasets, particularly single-cell RNA sequencing data, are high-dimensional, sparse, noisy, and dominated by zero values, making faithful low-dimensional representation challenging. Existing dimensionality-reduction methods may distort local neighbourhoods, global organization, or the cohesion of meaningful populations, with similar limitations arising in genealogical data. We introduce Contrastive Manifold Approximation and Projection (CosMAP), a graph-based unsupervised dimensionality-reduction method for producing faithful and interpretable embeddings. CosMAP extends the graph-based framework of UMAP by combining cosine-similarity neighbourhoods with temperature-normalized contrastive affinities, which are optimized in the embedding space using an attractive--repulsive objective. It further employs a two-phase refinement strategy: an intermediate higher-dimensional representation is first learned and then used to reconstruct the neighbourhood graph and initialize the final low-dimensional embedding. We evaluate CosMAP on MNIST and USPS handwritten-digit datasets, mouse retina and cortex single-cell RNA-sequencing datasets, and a large genealogical kinship dataset derived from BALSAC-CARTaGENE. Compared with state-of-the-art dimensionality-reduction methods, CosMAP produces more coherent visual representations, improves neighbourhood preservation, and provides clearer global organization of digit classes, biological cell populations, and regional genealogical patterns. These results indicate that CosMAP offers a robust framework for exploratory analysis of complex, sparse, high-dimensional data. The implementation is publicly available at https://github.com/FenosoaRandrianjatovo/CosMAP-dr.
Fenosoa Randrianjatovo, Maya Saleh, S. Girard et al.· 0 citations
Empirical evaluations on eight public real-world biomedical HDLSS gene-expression datasets demonstrate that GINet-DGC achieves competitive and stable performance against 17 baselines, and support the effectiveness of the proposed framework within the evaluated public biomedical HDLSS benchmark setting.
Xinran Zhang, Yang Sheng, Sijie Shen et al.· Applied Informatics· 0 citations
OmicFormer, a Transformer-based architecture that embeds two complementary statistical priors, i.e., feature-label associations and feature-feature dependencies, directly into its representation learning, provides an interpretable and generalizable foundation for omics-based precision medicine.
H. Jiang, C. Yang, M. Qin et al.· medRxiv· 0 citations
Motivation In this paper, we demonstrate that it is feasible to train a deep generative model for dimensionality reduction with millions of features using few samples, which makes this type of generative model a more versatile alternative to standard methods for dimensionality reduction. Specifically, we hypothesize that for a decoder-only model, the number of training samples required is almost independent of the feature dimensionality in most network architectures. Results Through an extensive set of experiments on synthetic non-linear data, we validate this hypothesis. We also train the model on a downsampled version of the 1000 Genomes Project (1KGP) dataset to further assess its behavior under controlled reductions in sample size. Furthermore, we train a deep generative decoder (DGD) on a curated dataset from the International Cancer Genome Consortium (ICGC), which contains 4.4 million features. It is trained on approximately 4,000 samples and tested on 1,000 samples. The resulting latent representation exhibits clear clustering, and when methods are reduced to the same number of dimensions, it outperforms PCA and VAE for tumor type classification. Additionally, the DGD is computationally efficient and can be trained on a 16GB GPU. Availability and implementation Code is available at https://github.com/cpancott/ReceptiveDGD. Contact corrado.pancotti@helmholtz-munich.de; akrogh@di.ku.dk Supplementary information Supplementary data are available with this preprint.
C. Pancotti, P. Fariselli, J. Meisner et al.· bioRxiv· 0 citations
Emergence of large scale protein language models (pLMs) has led to significant performance gains in predictive protein modeling. However, it comes at a high price of interpretability, and efforts to push representation learning toward explainable feature spaces remain scarce. The prevailing use of domain-agnostic and sparse encodings in such models fosters a perception that developing both parameter-efficient and generalizable models in a low-data regime is not feasible. In this work, we explore an alternative approach to develop compact models with interpretable embeddings while maintaining competitive performance. With the bidirectional long short-term memory autoencoder (BiLSTM-AE) model trained on positional property matrices, we introduce a soft weight matrix nondiagonality penalty and a one-hot encoded sequence clustering-based contrastive loss. As evidenced by Jacobian analysis, the penalty aligns embeddings with the initial feature space, whereas the contrastive loss organizes the latent space semantically. This combination leads to consistent improvements in performance on a suite of eight common peptide biological activity and physicochemical properties benchmarks. The use of amino acid physicochemical properties and density functional theory (DFT) derived cofactor interaction energies as input features provides a foundation for intrinsic interpretability, which we demonstrate on fundamental peptide properties. The resulting model is over 33,000 times more compact than the state-of-the-art pLM ProtT5. It demonstrates performance stability across diverse benchmarks without task-specific fine-tuning, showcasing that domain-tailored architectural design can yield highly parameter-efficient models with fast inference and preserved generalization capabilities.
Evgeniy Nam, Y. Din, Nikita S. Serov· Journal of Chemical Informat...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
