Long-horizon reasoning and agentic workloads strain KV-cache capacity and decode attention throughput. We present Jet-QSparse, a training-free method that combines compact NVFP4 cache storage with native FP4 query-key computation on Blackwell tensor cores. A dense scan scores the full key history and selects 128-token...
Hao-Zhan Tang, Hao Kang, Meng Lu et al.· 0 citations
This paper introduces Weakening-Induced On-Policy Self-Distillation (WISD), which instead constructs a weaker counterpart of the original model and uses the contrast between the two as a training signal without requiring privileged information.
Lightning OPD is proposed, an offline on-policy distillation framework that enforces teacher consistency and eliminates the need for a live teacher server entirely and achieves comparable performance to standard OPD while delivering 4.0x higher training efficiency.
Ye-Cheng Wu, Song Han, H. Cai· arXiv.org· 30 citations· ⚡2
Lightning Weave is introduced, a post-training framework that extracts and composes independently learned capabilities in a single student through on-policy distillation and achieves a state-of-the-art accuracy-efficiency frontier.
Ye-Cheng Wu, Song Han, Han Cai· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.