ACTD: Anchor-Based Cross-Tokenizer Distillation with Residual Regularization
An Anchor-Based Cross-Tokenizer Distillation with Residual Regularization with Residual Regularization (ACTD) bridges structural heterogeneity through vocabulary and sequence alignment, while mitigating alignment noise via a novel anchor loss with residual regularization.
Huiyi Zhang, Zijian Li, Xiaocheng Feng et al.
· 0 citations