FragBERTa is introduced, a molecular fragment-aware transformer-based representation language model pretrained using masked language modeling on Sequential Attachment-based Fragment Embedding (SAFE) representations, suggesting that fragment-based string representations offer advantages over atom-level representations for scaffold-sensitive and interaction-driven tasks.
Abstract
Automated exploration of chemical space through machine learning has become a central strategy in modern drug discovery, where the ability to learn transferable molecular representations directly impacts predictive modeling performance. Recent advances in chemical language models have primarily relied on atom-level string encodings such as SMILES or SELFIES, treating molecules as sequences of individual atoms or characters. However, medicinal chemistry reasoning is often fragment-centric, emphasizing scaffolds, functional groups, and modular substructures rather than isolated atoms. In this study, we introduce FragBERTa, a molecular fragment-aware transformer-based representation language model pretrained using masked language modeling on Sequential Attachment-based Fragment Embedding (SAFE) representations. SAFE decomposes molecules into chemically meaningful fragments while preserving attachment information, enabling the model to learn fragment-level structural patterns within a sequence framework. FragBERTa is pretrained on 11 million unique SAFE sequences extracted from multiple public chemical databases and subsequently finetuned on diverse MoleculeNet benchmarks covering both regression (ESOL, FreeSolv, Lipophilicity, PDBbind) and classification (BACE, BBBP, HIV, Tox21, SIDER) tasks under both random and scaffold splits. Our evaluation shows that FragBERTa achieves competitive performance across tasks, often matching or surpassing prior models. In regression, it demonstrates strong gains in lipophilicity and the best scaffold-split performance on protein-ligand binding affinity within our comparison. In classification, FragBERTa attains the strongest reported random-split ROC-AUC results on BACE, HIV, and Tox21 and the strongest reported scaffold-split ROC-AUC performance on BBBP among the compared baselines, and delivers robust performance across bioactivity, permeability, and toxicity benchmarks, indicating that fragment-based representations effectively capture biologically driven and interaction-sensitive endpoints. Collectively, these findings suggest that fragment-based string representations offer advantages over atom-level representations such as SMILES and SELFIES for scaffold-sensitive and interaction-driven tasks, highlighting the value of fragment-level inductive bias for future molecular foundation models in drug discovery.
Molecular embedding models can serve as foundational infrastructure for computational chemistry and drug discovery, where reusable vector representations support property prediction, virtual screening, and retrieval. Most molecular encoders are specialist models built around a single molecular view, producing unconditional vectors with no language interface for varying the representation. We ask whether multimodal large language models (MLLMs), which natively process images, text, and symbolic inputs, can instead serve as \emph{general molecular embedding models} that produce embeddings conditioned on both a molecular profile and a natural-language semantic context. We introduce \textbf{MolEmb}, a lightweight framework that adapts MLLMs by aligning molecular profiles with textual descriptions in a shared embedding space using a bidirectional contrastive objective. The resulting embedding model is competitive on molecular property prediction and supports cross-modal molecule--text retrieval in the same space. We further introduce \textbf{MolCAR}, a diagnostic benchmark for context-aware retrieval, and find that context-aware molecular embedding is primarily a data property of the supervision. These results suggest that MLLMs are not merely chemistry assistants or generators, but a viable and extensible route to general molecular embedding models.
Xinjian Zhao, Xiangru Jian, Yaoyao Xu et al.· 0 citations
Structure-based generative models (SBGMs) hold great promises for accelerating drug discovery by enabling target-aware molecular design. However, existing approaches face fundamental challenges: three-dimensional graph-based models can explicitly incorporate protein structural information but often generate chemically implausible molecules due to limited training data, while chemical language models (CLMs) produce chemically plausible molecules but struggle to effectively leverage three-dimensional structural information for structure-conditioned generation and hard to incorporate lead optimization functionality due to the nature of SMILES string. Here, we present StructureSAFE, a structure-aware chemical language model that resolves this trade-off by integrating protein structural and evolutionary encoders with the SAFE molecular representation via pretraining and finetuning training scheme, enabling both de novo hit identification and a comprehensive suite of lead optimization subtasks within a unified framework. Comprehensive benchmarking on the MolGenBench dataset demonstrates that StructureSAFE achieves state-of-the-art (SOTA) performance across multiple metrics, with particularly pronounced improvements in chemical plausibility relative to graph-based models lacking pretraining. Evaluation on a rigorously constructed held-out test set further confirms its ability to generate drug-like, synthetically accessible molecules with competitive predicted binding affinities for previously unseen targets on both hit identification and lead optimization setting. In silico case studies across four therapeutically relevant targets validate its capacity to generate chemically plausible molecules that recapitulate key binding interactions of known high-affinity ligands while proposing novel interactions for potential better affinity and exploring previously unknown regions of chemical space. Taking together, StructureSAFE represents a versatile and practical tool to provide high-quality candidate molecules for augmenting medicinal chemistry workflows in both hit identification and lead optimization campaigns.
Bo Yang, Ke Xu, Chijian Xiang et al.· bioRxiv· 0 citations
Molecular property prediction is central to cheminformatics and environmental chemistry, where accurate modeling of physicochemical properties supports risk assessment and molecular design. Classical descriptors and recent advances such as ChemBERTa have enabled learning chemically contextual representations directly from SMILES, while the integration of structured descriptors with transformer-based embeddings offers a promising pathway toward accurate and interpretable prediction. In this study, we introduce Path-Weighted Atom Vectors (PWAVs), a descriptor family that captures atom-level, environment-aware structural information. We evaluate PWAV both as a standalone representation and in combination with ChemBERTa embeddings through a gated fusion architecture incorporating modality dropout, FiLM conditioning, and auxiliary supervision. Experiments on six physicochemical property datasets ( log P, log S, log BCF, boiling point, melting point, and vapor pressure) show that PWAV generally improves over classical fingerprint descriptors within learned models and achieves competitive performance relative to established external baselines on several endpoints. The strongest gains are observed for boiling point, aqueous solubility, and partition coefficient prediction, where descriptor-embedding fusion yields the best results among the learned models considered. Ablation analyses demonstrate that PWAV contributes complementary structural information beyond SMILES-only ChemBERTa representations, while SHapley Additive exPlanations-based interpretability shows that predictive signal is concentrated within a compact subset of features, enabling an efficient reduced representation (PWAV-64). Nested cross-validation further confirms the robustness of PWAV within the XGBoost framework. Overall, PWAV provides a compact, interpretable, and extensible descriptor framework that integrates effectively with modern representation-learning approaches. These results position PWAV as a competitive and chemically transparent component for hybrid molecular property prediction, rather than as a replacement for domain-specific benchmark systems.
M. Afzal, S. Siddiqi· Physica Scripta· 0 citations
Fragment-based drug discovery (FBDD) uses small molecular fragments as starting points for drug development, and machine-learning models that operate on molecular string representations are increasingly applied to fragment-related tasks. SMILES is the dominant such representation, but its paired ring closures and balanced parentheses introduce syntactic complexity that can affect model behavior. We present a controlled study of how molecular string representation influences transformer-based fragment recovery, using an FBDD-motivated label pipeline: reference fragments are derived from known drugs via RECAP fragmentation and docking-based ranking, and the model is scored on recovering them. We introduce DeepBERTa, a ChemBERTa-derived transformer pretrained on DeepSMILES, and evaluate it against a matched SMILES baseline under identical architecture, data splits, and optimization (approximately 34,000 drug–fragment pairs). DeepSMILES produces syntactically valid predictions more often than SMILES (54.2% vs. 43.5%) and a higher full test-set mean Tanimoto similarity (0.36 vs. 0.29). A per-sample selection between the two representations raises mean Tanimoto to 0.43, and a deployable variant that selects on model confidence rather than the reference recovers most of this gain. Among molecules where one representation strictly wins, DeepSMILES wins more often than SMILES (27.8% vs. 16.3% of test molecules). These results show that string representation meaningfully influences transformer fragment recovery and that representation-aware selection outperforms either representation alone.
Aayush Kothari, Amisha Gupta, Nisarg Shah et al.· Journal of Cheminformatics· 0 citations
Abstract Motivation Molecular property prediction is a fundamental task in drug discovery and plays a key role in accelerating the identification of promising candidates. Existing methods have primarily focused on atom–bond information for molecular representation learning. However, atoms and bonds are often not explicitly organized as coupled learnable entities, and prior knowledge such as functional motifs and global molecular attributes remains largely underutilized. In addition, they lack effective mechanisms to enable interaction and fusion across different structural granularities. Consequently, the resulting representations often capture only partial chemical semantics. Results To address these issues, we propose HMG-MRL, a unified framework for hierarchical multi-granularity molecular representation learning. This framework systematically integrates domain knowledge across three granularities: fine-grained atom–bond information, medium-grained functional motifs, and coarse-grained molecular attributes. Specifically, we introduce an atom–bond bipartite graph modeling approach that treats atoms and bonds as explicit learnable node types and jointly models atom–atom, atom–bond, and bond–bond local interactions within a unified propagation framework. In parallel, we integrate multiple substructure decomposition strategies to construct a diverse motif vocabulary and design a Motif Transformer that employs self-attention to capture global interactions among motifs. Moreover, we design a cross-granularity communication module to facilitate information exchange across granularities. Extensive experiments on nine public benchmark datasets show that HMG-MRL achieves competitive predictive performance. Case studies further indicate that the model can reveal key molecular components across different granularities, with its diverse substructure decomposition strategies offering complementary structural patterns. Availability and implementation The data and code of HMG-MRL are available at https://github.com/AKZstar/HMG-MRL.
Xing Zhao, Xianlai Chen, Yunbo Wang et al.· Bioinformatics· 0 citations
Large language models (LLMs) are widely applied across chemical tasks, such as molecular property prediction, which underpins drug discovery. Molecular LLMs represent a molecule through several modalities, notably a 1D SMILES sequence or a 2D molecular graph. Both encode molecular information implicitly, so the contribution of individual substructures remains opaque. Retrieval and augmentation methods add context, but from external sources. However, the cues chemists reason over are the internal substructures that drive a property up or down. We propose MR-MoL, a multi-granular rationale-guided molecular LLM that supplies this evidence directly. A fine-tuned GNN scores each substructure through masking, and the most influential ones are serialized as a ranked, direction-tagged rationale that the LLM reads alongside the SMILES sequence and molecular graph. The rationale spans three levels of granularity: Murcko scaffolds with their side chains, BRICS fragments, and functional groups. This is, to our knowledge, the first method to expose GNN-derived attributions to an LLM as evidence for property prediction. On eight MoleculeNet tasks, MR-MoL achieves the best overall results among generalist models and narrows the gap to specialist models tuned for each task. Five diagnostics further confirm that the model reads the rationale rather than merely benefiting from its presence. Its direction, rank, and substructure each shape the prediction, and its attributions reproduce known structure-property relationships.
Junwoo Park, Minyoung Shin, C. Lee et al.· 0 citations