Universal Redundancies in Time Series Foundation Models
This study develops a theoretical framework framing transformers as kernel regressors, motivating a purely intrinsic strategy for ablating heads based on the stable rank of the per-head projection matrices, and uncovers the specific heads responsible for degenerate phenomena widely observed in TSFMs.