We have 3 of 30 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Retrieval-based speculative decoding (SD) drafts tokens by copying continuations from existing text, which suits coding agents that repeatedly reproduce code, logs, and earlier attempts. Yet existing methods fall short in agent pipelines: much of the reusable text is missing from their corpora or stored in a form that...
From a request's prefill expert activations, ELDR builds an expert signature predicting the experts it will activate during generation, which reduces median TPOT by 5.9-13.9% over the strongest of four load-balancing baselines across three MoE models and two workloads.
OasisKV is presented, a memory-centric LLM inference system design that alleviates HBM capacity pressure by decoupling full KV-cache storage from HBM during LLM decoding and observes that future important tokens can be predicted accurately in advance using lookahead tokens drafted by speculative decoding (SD).
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
