MS-GLA: Multi-Scale Gated Linear Attention for Addressing Representational Bottlenecks via Multi-Temporal Resolution
MS-GLA consistently achieves higher accuracy and lower perplexity than GLA at matched parameter counts, validating multi-temporal resolution decomposition as a principled and effective extension of Gated Linear Attention.
Prasoon Dev, Anirudh Sankar, Vasudeva Varma
· 0 citations