MechaTerp-TRACE: A Novel Approach for Component Ablation Analysis in Language Models
It is found that the components carrying the most effect are the same few, positionally fixed components in every model, regardless of which entity a prompt asks about, and that once these are removed, the remaining support is close to evenly spread in eleven of the thirteen models.
B. Colelough, Davis Bartels, Madeline Bittner et al.
· 0 citations