Parnet is a multi-task foundation model trained end-to-end on 223 eCLIP-seq experiments spanning 150 RBPs to predict base-resolution RBP binding profiles directly from RNA sequence, and establishes the RBP interactome as a compact, functionally sufficient, and interpretable basis for foundation model pretraining in RNA biology.
Abstract
RNA-binding proteins (RBPs) orchestrate a complex combinatorial regulatory “code” that governs RNA splicing, stability, localization, and translation. Learning the relationship between RNA sequences and these processes is a central challenge in genomics. Foundation models, notably RNA language models, have emerged as the dominant approach, learning general-purpose representations from unlabeled sequence at scale. While RNA language models have demonstrated impressive performance across a broad range of downstream tasks, they generally learn from sequence reconstruction objectives alone, lacking direct connections to the regulatory principles that govern RNA function. Here we introduce Parnet, an RNA foundation model trained directly and exclusively on experimental CLIP-seq data. Parnet is a multi-task foundation model trained end-to-end on 223 eCLIP-seq experiments spanning 150 RBPs to predict base-resolution RBP binding profiles directly from RNA sequence. This CLIP-seq pretraining strategy departs fundamentally from the masked-language-modeling paradigm, anchoring learned RNA representations directly in measured protein–RNA interactions rather than sequence statistics. Parnet substantially outperforms its single-task predecessor RBPNet in binding profile and motif recovery, generalizes to unseen cell types and iCLIP data, and recapitulates position-dependent splicing regulation. Frozen Parnet embeddings, without task-specific fine-tuning, match or exceed the performance of both task-specific tools, as well as larger self-supervised RNA and genomic language models across diverse downstream tasks, including RNA biotype classification, lncRNA chromatin localization, translational efficiency, splice-site recognition, intron retention, and non-coding variant effect prediction. Importantly, Parnet remains mechanistically interpretable, tracing predictions back to the specific RBPs and motifs that drive them. These results establish the RBP interactome as a compact, functionally sufficient, and interpretable basis for foundation model pretraining in RNA biology.
Much of the human genome’s non-protein-coding fraction acts directly through RNA, yet the structural and functional roles encoded in these sequences remain poorly understood. Applying deep learning is hindered by scarce RNA structural data and it remains unclear what biological constraints such models can recover directly from the abundant RNA sequences alone. Here, to address these challenges, we developed NucleicBERT, a self-supervised masked-language model that learns contextual representations from single sequences without evolutionary information. Explainable artificial intelligence analyses show that the model organizes RNA sequences in latent space and encodes structural properties indicating that biologically meaningful constraints are learned from sequence correlations alone. When fine-tuned for downstream structural and functional tasks, NucleicBERT requires only single sequences while matching or exceeding current RNA prediction models. This alignment-free framework addresses the scarcity of annotated 3D RNA data while providing a rapid, computational complement to experimental techniques. By bridging abundant unlabelled sequence data with scarce structural annotations, NucleicBERT advances RNA structure prediction and informs how large language models encode biological information.
Utkarsh Upadhyay, Julian Herold, Markus Götz et al.· Nature Machine Intelligence· 0 citations
Abstract While protein–RNA interactions are fundamental to post-transcriptional processes, achieving a holistic understanding of their regulatory logic remains challenging. Current computational models often treat binding affinity, interface mapping, and RNA design as isolated tasks, thereby failing to provide a unified perspective of the protein–RNA interactome. Here, we introduce ProRB, a unified sequence-based framework that jointly estimates protein–RNA binding affinity, predicts binding interfaces in proteins and RNAs, and generates protein-binding RNA sequences from protein sequences. By fusing protein and RNA embeddings from language models via adaptive cross-modal attention, ProRB learns contextual and relational features for predicting protein–RNA binding affinity and interface contacts, outperforming or achieving competitive performance compared to structure-based methods. Notably, its cross-attention maps reveal interpretable, motif-centric binding logic hidden in protein–RNA interactions. Building on this interpretability, ProRB enables computationally prioritized design of protein-binding RNA sequences with enhanced biophysical properties and functional motifs. By unifying the prediction, interpretation, and generation tasks, ProRB provides a scalable unified model for decoding the protein–RNA interaction and engineering motif-guided RNA therapeutics.
RNA motifs and Low-Complexity Repeats (LCRs)—recurrent, conserved sequences of nucleotides—serve as the fundamental vocabulary of RNA structure and biological function. While current RNA foundation models have revolutionized sequence modeling through Transformer architectures, they predominantly prioritize modeling global dependencies within the raw sequence. To address this limitation, we propose MotRNA, a motif-aware RNA language model integrating an Explicit N-gram Memory mechanism. Unlike standard implicit neural modeling, MotRNA employs a conditional memory module to explicitly retrieve and fuse motif embeddings based on input k-mers. This mechanism provides direct access to strictly defined local patterns, complementing the global context modeled by self-attention. We validate MotRNA on large-scale RNA datasets. The model achieves a 9.6% absolute improvement in Masked Language Modeling (MLM) accuracy at a 30% masking ratio compared to state-of-the-art baselines. Notably, MotRNA maintains high robustness under extreme masking ratios, indicating effective biological signal reconstruction from sparse context. Crucially, our analysis reveals that the backbone encoder becomes self-contained post-training: the memory module facilitates the internalization of motif semantics into the model parameters, allowing the backbone to retain performance advantages even when the memory is detached during inference.
Xiang-Yu Ji, Xin Wang, Yang Zhang et al.· Proceedings of the 32nd ACM...· 0 citations
Background RNA inverse folding designs nucleotide sequences expected to adopt a prescribed secondary structure. Search-based solvers can optimize folding-model objectives effectively, but difficult targets can require extensive sampling, and structural optimization alone does not explicitly preserve the sequence distributions or conserved motifs of natural RNA families. Methods We developed RIFT-VAE, a Transformer-based conditional variational autoencoder that receives a context-free grammar parse-tree representation of a target secondary structure and generates nucleotide labels on the corresponding tree. The framework combines progressively richer grammar rules, self-refinement learning from generated structure-sequence pairs, and cross-entropy-method optimization in the learned latent space. We evaluated RNAfold minimum-free-energy agreement on an RNAcentral-derived test set and the EteRNA100 benchmark, compared the method with four search-based solvers under matched total time budgets, and examined GC-content control and covariance-model family annotation. Results The complete pipeline achieved RNAfold-Correct/RNAfold-MCC values of 0.833/0.994 on the RNAcentral-derived test set and 0.760/0.977 on EteRNA100. Latent-space optimization accounted for the largest increase in exact structural recovery. Under a 3,600-s total budget on EteRNA100, sequences generated by RIFT-VAE improved the exact-match rate of every tested downstream search method when used as warm starts; the largest change was observed for RNAInverse (Correct, 0.297 to 0.803; MCC, 0.505 to 0.985). The pretrained model also produced sequences with measurable correct-family covariance-model hits and supported explicit GC-content conditioning. Conclusions RIFT-VAE is best interpreted as a hybrid generative-search framework: pretraining supplies a structure- and family-informed proposal distribution, whereas latent optimization concentrates evaluations in high-scoring regions. The reported structural scores are specific to RNAfold minimum-free-energy validation and do not establish biochemical function. Orthogonal folding predictors, stricter homology-controlled splits, diversity-aware evaluation, architecture-matched dot-bracket ablations, and experimental assays remain priorities for validation.
These approaches improve generalisability, reduce reliance on deep evolutionary information, and enable proteome-scale prediction of RNA-binding residues, providing a route to map and interpret the molecular logic of protein-RNA interactions.
Rozeena Arif, Alfredo Castello· Current Opinion in Structura...· 0 citations
Abstract Deciphering sequence–function relationships in long non-coding RNAs (lncRNAs) remains challenging due to rapid evolutionary turnover and limited primary sequence conservation. Alignment-based approaches often fail to detect functional domains, and fixed-length k-mer models inadequately capture variable-length regulatory elements. Here, we introduce RNA-Lexis, a non-parametric statistical framework for unbiased discovery of candidate RNA sequence elements. RNA-Lexis applies segmentation based on local conditional probabilities to identify non-random sequence extensions, enabling detection of recurrent, variable-length motifs without prior biological assumptions. Conceptually analogous to language segmentation, the framework partitions continuous RNA sequences into statistically defined units (“xmotifs” and “cores”), providing an interpretable representation of sequence architecture. RNA-Lexis reconstructs the modular organization of well-characterized lncRNAs, including XIST and NORAD. In additional case studies, RNA-Lexis prioritized recurrent GC-rich elements in SNHG14 that were tested experimentally and shown to bind histones in RNA pulldown assays. RNA-Lexis also identified recurrent LINC01001 core motifs that overlap chromatin interaction patterns detected by GRID-seq. These analyses support the use of RNA-Lexis to nominate candidate sequence elements for functional follow-up, while biological function remains dependent on orthogonal experimental validation. RNA-Lexis provides a statistically grounded and interpretable framework for motif-level analysis of lncRNAs. Rather than directly inferring function, the method identifies recurrent sequence architecture and prioritizes candidate elements for mechanistic testing.
Haim Y. Bar, Amit Felach, A. Bester· Nucleic Acids Research· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
