Morphological Hijacking in Frozen Language Models: Recovering Structured Representations and Testing the Limits of Algebraic Composition

Abstract

This revised manuscript presents a controlled study of morphological hijacking in frozen language-model representations using the synthetic LUXVAR/VARZIN framework. The study shows that adversarial surface structure can dominate frozen representations, while a lightweight trained projection head can substantially recover the targeted categorical/group-position structure under controlled held-out, cross-script, and out-of-distribution tests. The revision incorporates the complete Level-3 composition diagnostic chain. A linear decoder failed on seen-pair composition, after which diagnostics tested optimization, structural rank deficiency, cross-family geometric alignment, and decoder capacity. A pre-registered one-hidden-layer MLP (64 hidden units, ReLU) fit the 120 seen ordered pairs almost perfectly (mean L3-A accuracy = 0.996 across five seeds), showing that decoder capacity was sufficient to fit the training set. Crucially, the same MLP did not generalize to 24 held-out ordered pairs: TRUE accuracy was 0.106 (38/360), compared with 0.281 (101/360) for the shuffled-label control and 0.175 (63/360) for the wrong-operation control. The revised interpretation is deliberately narrow. The intervention provides evidence for recovery of the targeted categorical/group-position structure, but it does not demonstrate systematic algebraic composition. The unseen-pair result is consistent with memorization/interpolation rather than learned application of the (i+j) mod 12 rule. The paper therefore treats composition as an important negative boundary condition, not as evidence that composition information is universally absent from the underlying representations. Conclusions are restricted to the tested synthetic lexicon, GPT-2-small extraction pipeline, preprocessing, alignment procedures, and decoder classes