Erased, Rerouted, or Rescaled? Post-Training and the Causal Quotient of a Language Model's Belief State
What happens to information a pretrained model already encodes when post-training no longer rewards using it? The common language of representation compression conflates three fates: information may be erased, rerouted away from the decision while still represented, or rescaled to occupy less variance while still repre...
Wei-Han Li, Tianshi ZHENG, Jun-Hao Wu et al.
· 1 citation