Asymmetric Structural Transfer Between Natural Language and Biological Foundation Models

Abstract

Cross-domain transfer is a defining property of foundation models, yet whether such transfer is symmetric across domains remains unknown. Prior work has reported a striking transfer from natural language to biological sequences: language models fine-tuned only on English structural tasks acquire zero-shot protein-homology discrimination. Here we ask the converse and general question—is structural transfer between language and biology directional? —and answer it systematically. We first reproduce forward transfer (language →biology) under controlled conditions, then evaluate the reverse direction (biology →language) across fine-tuning, iso-token continued pretraining, model scaling, multiple biological foundation-model families (ESM-2, ProtBERT), and adversarial synthetic structure tasks. Reverse transfer is consistently weak: it does not exceed matched-token controls, does not scale, and does not generalize. In an architecture-matched 2 × 2 analysis on models with known training data—which eliminates the pretraining-contamination confound that clouds large-model studies—a language model retains far more competence when moved to biology (off-domain drop 0.08) than a biological model retains when moved to language (drop 0.36). Scaling widens rather than closes this gap: language →biology transfer strengthens with size while biology →language transfer decays toward chance, a pattern shared by two independent protein-model families. Our findings establish that shared structural regularities between natural language and biological sequences do not imply symmetric representational transfer, revealing an intrinsic directionality in cross-domain foundation-model learning.