Log in
Back to feed

Similar papers

Book Open access Aug 2026

UniEdit: A Graph-based MoE Alternative to Sequence Generation for Molecular Editing

This paper proposes UniEdit, a Unified Graph-based Mixture-of-Experts (MoE) Molecular Editing model that offers a robust alternative to LLMs and incorporates a Mixture-of-Experts architecture that dynamically routes tasks to specialized components.

Jiajun Yu, Zhihao Wu, Yizhen Zheng et al. · 0 citations
Book Open access Aug 2026

Caduceus: MoE Foundation Models for Unifying Biological and Natural Language

This paper introduces Caduceus, a family of MoE-enhanced foundation models built with a hierarchical pre-training paradigm to jointly integrate biological and natural language, and incorporates a multi-task instruction tuning phase, enabling robust protein parsing and natural language question answering.

Mingze Yin, Yiheng Zhu, Jialu Wu et al. · 0 citations
Preprint Aug 2026

Bi-semantic Chemical Embedder for Joint Representation Learning of SMILES and Natural Language

Transformer models have revolutionized natural language processing (NLP), and text-based molecular representations like SMILES have successfully extended these architectures to chemistry. However, domain-adaptive pre-training often causes models to overfit to chemical syntax, catastrophically forgetting their foundational semantic capabilities. To address this challenge, we introduce CheMatE, a chemistry-oriented embedding model that jointly captures molecular structure and domain-specific natural language within the same representation space. Built on a ModernBERT backbone, CheMatE learns bi-semantic representations through a two-stage training procedure: continued masked language modeling (MLM) followed by a Matryoshka contrastive learning stage via Multiple Negative Ranking Loss (MNRL). First, we train the model using MLM on a novel, large-scale corpus of SMILES-annotated, long-context scientific documents that were constructed and curated from FineWeb and ChemPile (comprising 10.4B and 11.5B tokens, respectively). Subsequently, the model undergoes contrastive learning using a synthetic dataset of SMILES-text pairs algorithmically derived from our original training corpus. This design exposes the model to SMILES-enriched scientific literature, enabling bi-semantic understanding. We evaluate CheMatE across a range of downstream tasks covering molecular property prediction and scientific language understanding. Our results demonstrate that coupling our custom-curated datasets with this sequential training strategy yields robust, highly transferable representations. By effectively unifying structural and contextual signals within a single text-based framework, CheMatE achieves competitive performance across both specialized chemistry models and general-purpose language model baselines.

David Ming Segura, Jeremy Goumaz, Joshua W. Sin et al. · 0 citations
Aug 2026

Speaking the Native Language of LLMs: A Discrete Architecture for Molecular Comprehension.

MolLingua, a token-centric, dual-modal framework designed for native molecular understanding, uses a dual-branch Residual Vector Quantization engine to discretize heterogeneous, high-dimensional spatial 2D and 3D features into compact code sequences rather than relying solely on continuous projections.

Haoyang Liu, Xikang Feng, Fei Guo et al. · 0 citations
Open access Jul 2026

Navigating chemical-linguistic sharing space with heterogeneous molecular encoding

HME is presented, a framework that combines multiple views of molecules to improve molecular understanding and design and enables bidirectional navigation of the chemical-linguistic space, achieving consistent improvements across molecular comprehension and design tasks over strong baselines.

Liuzhenghao Lv, Hao Li, Yu Wang et al. · 0 citations
Preprint Jul 2026

Beyond Scale and Generation: Understanding Language Model-based Entity Matching

The factors underlying performance differences across matcher architectures are clarified and motivate future research and benchmark designs that better disentangle architectural choices from model-level factors while explicitly evaluating distribution shift and cross-dataset transferability.

Zeyu Zhang, Xue Li, Iacer Calixto et al. · 0 citations