AS$^2$D: Accelerating On-Demand Audio Understanding on Mobile Devices
Speculative decoding accelerates autoregressive generation by using a smaller drafter to propose tokens for batched verification by a larger target. However, conventional speculative decoding couples drafting to the target's evolving verified prefix, serializing drafting and verification. We ask whether this dependency...
Yun-Zhe Li, Kyoungjun Park, Hong-Zi Zhu et al.
· 0 citations