TSS: Target-Side Sparsification for Speculative Decoding in Domain-Specific Large Language Models
TSS, a target-side sparsification framework for speculative decoding that employs an acceptance- and metric-aware breadth search to explore multi-layer skip configurations without imposing a fixed priority between the two objectives, is proposed.
Hai-Bo Hu, Lian-Ming Huang, Qiao Li et al.
· 0 citations