Protein language models (pLMs) such as ESM-2 achieve strong zero-shot mutation-effect prediction, yet the internal computations supporting these predictions remain poorly understood. We introduce a sparse feature circuit framework that combines sparse autoencoders, integrated-gradients attribution, and activation patching to identify the latent features that causally mediate zero-shot mutation effect prediction in ESM-2 650M. We evaluate this framework over 67 mutations ranging from strongly deleterious to weakly deleterious in the DNAJA1 J-domain, where ESM-2 predictions agree strongly with deep mutational scanning measurements. We find that circuits selected by indirect effect recover the model’s predictions more efficiently and provide more informative biological explanations than those selected by raw activation changes, showing that activation magnitude does not necessarily reflect causal importance. We find that related substitutions reuse substantial portions of their recovered circuits, ranging from 40% to 75%, and that the shared features often represent residues in three-dimensional contact with the mutation site. To our knowledge, our work provides the first causal, feature-level account of zero-shot mutation effect prediction in a pLM.
Saishradha Mohanty, Manya Phutela, A. G. Green· bioRxiv· 0 citations
Active learning provides an efficient strategy for molecular property prediction by iteratively prioritizing compounds for experimental evaluation. However, the effectiveness of active learning pipelines depends strongly on the choice of molecular representation, and systematic understanding of how representation families affect the active learning process in terms of uncertainty and predictive performance remains limited. In this work, we introduce ActiveFusion, a framework for integrating heterogeneous molecular representations within active learning workflows for molecular property prediction. The framework enables systematic evaluation of physicochemical descriptors, molecular fingerprints, learned graph neural network (GNN) representations, and pretrained Transformer-based representations, as well as feature-level fusion strategies that combine complementary chemical information sources. ActiveFusion evaluates models across four molecular property prediction regression tasks. Across datasets, we demonstrate that feature fusion between learned graph representations and physicochemical descriptors consistently improves prediction performance and discovery (average final iteration R2 of 0.71, 0.67, and 0.54 for our overall best representations Chemprop+RDKit, Chemprop, and RDKit, respectively). We show that exploration-driven acquisition strategies enhance scaffold coverage and promote sampling of structurally novel regions of chemical space, and that model-agnostic acquisition of new compounds based on diversity has strong performance. Notably, both pretrained and finetuned Transformer-based embeddings do not consistently outperform physicochemical features or GNN-learned representations in our setting, highlighting the continued relevance of chemically interpretable features and learned features from supervised, task-specific models for active learning applications in molecular property prediction. Overall, ActiveFusion provides a systematic framework for studying representation-acquisition interactions in molecular discovery with representation fusion capabilities. Our study offers practical guidance for designing active learning pipelines that balance prediction accuracy with chemical space exploration.
Nelson Evbarunegbe, Shiyun Wa, Luke Taylor et al.· Journal of Chemical Informat...· 0 citations
Goal-directed optimization is essential for steering molecular generators to propose candidates with desired properties. However, it is often implemented with policy-gradient reinforcement learning, which requires a generation-trajectory log-probability whose form depends on the model architecture and generation procedure. This makes an optimizer difficult to reuse across architectures and conditional generative designs. Supervised fine-tuning needs none of that machinery, but its update is driven by a fixed dataset, so the reward never enters the update. We introduce Elite-Weighted Supervised Fine-tuning (EW-SFT), which uses reward to guide elite selection of high-scoring molecules, and updates the model by its own pretraining loss on that set. Ablations show that reward information is passed primarily through elite selection, rather than through continuous weighting within the selected set. Because the update consumes only scored molecules and the model's native loss, the same rule applies across autoregressive, masked-diffusion, and discrete-flow generators, and across de novo, motif-extension, and linker-design tasks. Under a fixed budget of 3D shape alignment oracle calls on two kinase reference compounds, EW-SFT consistently outperforms the corresponding native optimizers. It further improves goal-directed optimization under a 2D similarity oracle on four held-out references and achieves comparable performance on a sample-efficiency benchmark without a trajectory-level RL formulation. These results demonstrate that EW-SFT is a unified and effective optimizer across molecular generators, design constraints, references, and oracles.
Shiyun Wa, Yifei Wang, A. G. Green et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
