Efficient Adaptation of Chemical Language Models for Molecular Property Prediction

Abstract

Chemical language models (CLMs) like ChemBERTa and Molformer enable compound property prediction, but their computational demands limit adoption in resource-constrained settings. We integrate Low-Rank Adapters (LoRA) with CLMs to significantly reduce trainable parameters required for finetuning. Results: We observe 3–5% area under the receiver operating curve (AUC) improvement across classification tasks for molecule toxicity, blood-brain barrier permeability, and flavor prediction over Molformer-XL. Our approach achieves Matthews correlation coefficient (MCC) scores of 0.80–0.90 across three tasks while reducing model parameters by 75–95%. By comparing embeddings from zero-shot and finetuned CLMs combined with molecular physicochemical properties, we attain optimal performance across four datasets. This lightweight adaptation retains performance efficiency while reducing over-parameterization.