Communication-Efficient LLM Adaptation over Decentralized GPU Meshes
It is shown that masking supports post-pretraining adaptation at high compression rates when anchored this way, and a spectral correction optimizer is introduced that uses these delayed anchor priors to denoise masked gradients without blocking the fast stream.
Sameera Ramasinghe, S. Siriwardhana, Thalaiyasingam Ajanthan et al.
· 0 citations