TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding
TreeGraft is a multi-drafter framework in which drafters of different costs jointly construct a shared draft tree that outperforms the better of the two fixed single-drafter endpoint strategies by 15.1% on average and reaches a maximum gain of 26.6%.
Jiaming Fan, D. Cao, Can-Chen Huang et al.
· 0 citations