Skip to content

RIBOSPAN: A Long-Context RNA Foundation Model for Versatile RNA Modeling

Aug 2026 · 0 citations · 29 references
Computer Science Biology

TL;DR

Frozen RNA-type evaluations show that RIBOSPAN learns state-of-the-art RNA representations, with a particularly clear advantage on long RNAs, and emerges as the strongest encoder-only RNA foundation model, achieving state-of-the-art performance in both full-transcript biological property prediction and zero-shot mutation-fitness modeling.

Abstract

Full-length RNAs, particularly messenger RNAs, often exceed the context lengths used to pretrain existing RNA foundation models, limiting complete-transcript modeling at single-nucleotide resolution. We present RIBOSPAN, a 1.61-billion-parameter bidirectional RNA foundation model natively pretrained with context lengths up to 10,240 nt. RIBOSPAN combines dense bidirectional self-attention, single-nucleotide tokenization, and attention-isolated sequence packing to enable high-resolution modeling of complete long RNAs. Native 10K pretraining preserves strong reconstruction at 10,240 tokens and, in a controlled long-context benchmark, maintains strong contextual responsiveness and context-specific representation separation while keeping perturbation-induced changes highly localized. Inference-time YaRN scaling recovers much of the contextual organization lost by direct short-context extrapolation, but induces substantially greater distal representation diffusion. Frozen RNA-type evaluations show that RIBOSPAN learns state-of-the-art RNA representations, with a particularly clear advantage on long RNAs. Across downstream biological benchmarks, RIBOSPAN emerges as the strongest encoder-only RNA foundation model, achieving state-of-the-art performance in both full-transcript biological property prediction and zero-shot mutation-fitness modeling. Building on the same backbone, we develop a multidimensionally conditioned discrete-diffusion framework for full-length mRNA generation and redesign, including synonymous-codon diffusion for protein-preserving CDS optimization. Together, RIBOSPAN establishes a powerful long-context foundation for transferable RNA representation learning, biological prediction, and full-transcript mRNA design.

View source

Similar papers

#machine learning Preprint Oct 2026

RNADyn: A Benchmark for Generating and Understanding RNA Dynamics

Ribonucleic acid (RNA) functions through conformational changes that are not fully captured by static structures. However, large-scale standardized RNA dynamics data remain limited, and existing approaches typically treat trajectory generation and dynamics understanding as separate objectives. Here, we introduce RNADyn...

Yi-Ming Huang, L. Bastian, Hanqun Cao et al. · 0 citations
Open access Sep 2026

UFold-X: an enhanced Dual & Dynamic U-Mamba model for long-range RNA secondary structure prediction

UFold-X, a dual-branch deep learning framework that combines a convolutional encoder for local structure modeling with a Mamba-based Visual State Space Module for capturing long-range dependencies, is presented, achieving the best performance among deep learning-based methods.

Lai-Yi Fu, Jia-Chun Li, Rui-Qi Wang et al. · 0 citations
Open access Sep 2026

NucleicBERT interprets RNA sequence space through self-supervised language modelling

NucleicBERT is developed, a self-supervised masked-language model that learns contextual representations from single sequences without evolutionary information that advances RNA structure prediction and informs how large language models encode biological information.

Utkarsh Upadhyay, Julian Herold, Markus Götz et al. · 1 citation
Open access Sep 2026

RNASeek: A Cross-Phyla Generative Foundation Model for Multipurpose RNA Modeling and Reinforcement Learning-Based Design

RNASeek establishes a unified pretrain–predict–optimize framework that connects learned RNA function to controllable de novo sequence design and provides a general strategy for engineering regulatory RNAs with desired properties.

Shi-Yuan Chen, Neil R. Fernandes, Wei Vivian Li et al. · 1 citation
#machine learning Preprint Oct 2026

Scaling RNA Foundation Models through Iterative Refinement

Eco-RNA is presented, an 80M-parameter RNA foundation model that makes inference depth adjustable through learned iterative refinement that surpasses the current SOTA Evo 2 40B on all three aggregate fitness metrics despite 500x fewer parameters.

Peng-An Chen, Liang Hong, Xin-Yuan Liu et al. · 0 citations
Open access Sep 2026

Conditional Generation and Inpainting of Non-Coding RNA Sequences with Masked Discrete Diffusion

Designing functional non-coding RNA (ncRNA) is fundamental to synthetic biology and RNA therapeutics, yet generative modelling for ncRNA has received far less attention than protein design. We present RNA-MDLM, a framework that extends Masked Discrete Language Models (MDLM) to the conditional generation and inpainting...

Utkarsh Upadhyay, Chuan-Kai Dai, Julian Herold et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.