Dynamic Semantic Compression for Efficient Latent-Space Inference in Large Language Models
A Dynamic Semantic Extraction and Inference (DSEI) framework, which achieves segment-level inference within the latent space through a two-stage training strategy, and substantially reduces both input and generation sequences and significantly enhances inference efficiency.
Pei-Pei Li, Dong-Sen Zhang, Yu-Chen Liu et al.
· 0 citations