BR-MoE, a memory-budget-aware co-design framework that treats per-expert bit-width and compensator rank as joint variables under a single memory constraint, establishes a superior memory–accuracy trade-off over state-of-the-art quantization methods, delivers substantial end-to-end inference speedups, and re-derives an...
Lei Jiang, Jing-Wei Sun, Jun-Qing Lin et al.· Proceedings of the Internati...· 0 citations
The Mixture-of-Experts (MoE) architecture has become the dominant paradigm for scaling Large Language Models, yet its explosive parameter growth routinely overflows the GPU memory budget. Weight quantization offers a practical way to fit these models into limited GPU memory, but reducing precision below 4 bits causes s...
Lei Jiang, Jingwei Sun, Jun-Qing Lin et al.· Proceedings of the Internati...· 0 citations
Lua-LLM is proposed, a learning-based global pruning framework that explores the optimal unstructured sparsity allocation and achieves flexible allocation for both layer-wise and intra-layer sparsity.
Ming Lu, Jing-Wei Sun, Jun-Qing Lin et al.· Neural Information Processin...· 1 citation
FlashQuant fuses the dense GEMM and sparse outlier SpMM paths into a single GPU kernel, enabling on-chip reuse of activation and output tiles across heterogeneous computations, and introduces three key techniques: sparse-dense tiling, which aligns outlier processing with dense GEMM tiles, and pipelined scheduling, whic...
Jun-Qing Lin, Jing-Wei Sun, Zhengding Hu et al.· 0 citations
DeltaLog is presented, a recurrent-state decoding scheme that reduces this overhead without changing the model semantics, and is implemented for GDN, KDA, and RWKV6 and integrated into a prototype serving stack.
By making DWC compute-efficient, Flash-DWC promotes the use of larger and more expressive filters and extends forward and backward propagation to both forward and backward propagation for efficient end-to-end training.
Zhiyi Zhang, Yang Zhao, Jing-Wei Sun et al.· ACM Transactions on Architec...· 0 citations
EMO combines graph analysis and pack-level models to formulate energy optimization as a constrained combinatorial problem, efficiently solving for optimal frequency policies under given latency targets.
Jiyu Luo, Shaoyu Chen, Jingwei Sun et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
