STEPQuant is proposed, a spatial-temporal post-training quantization framework for Delta-rule recurrent states that allocates precision according to error magnitude and memory lifetime, and jointly fits key-row and value-column scales based on state distributions and key-row impact on output error.
Bing-Chen Yao, Hao-Bo Xu, Hao-Kun Lin et al.· 0 citations
First, it is demonstrated that quantization is significantly more effective in preserving trustworthiness compared to pruning, and more importantly, it is demonstrated that compressing a reliable large model via quantization can produce SLMs with superior trustworthiness and adaptability compared to using small models...
Hao-Kun Lin, Kai-Jie Zhu, Hao-Bo Xu et al.· 2 citations
PILL (Probing-based InfiLling with preset-Length-free decoding), an efficient infilling method for DLMs that requires no preset initial length and adds far fewer extra forward passes than baselines, substantially reducing inference time is proposed.
Hao-Bo Xu, Si-Rui Chen, Yuan-Chen Bei et al.· 4 citations
This work proposes method that unifies textual reasoning and graph message passing within a masked diffusion language model, a language model with bidirectional attention and generative decoding that outperforms graph neural networks, graph transformers, and LLM-based baselines on all three TAG benchmarks across two ta...