We have 5 of 21 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
OLED-MoE is proposed, an expert offloading system that uses confidence-guided inter-iteration prediction to retain high-value experts in GPU memory without introducing extra prefetch traffic and compensates unavoidable cache misses through CPU-GPU cooperative execution.
PatchKV, a profile-guided recovery system for suffix-preserving revisions, decomposes adjacent context versions into an exact prefix, an updated span, and an aligned suffix, and achieves a speedup in mean resume time-to-first-token over full suffix recomputation.
Serverless edge computing, despite its flexibility and efficiency, is hindered by high startup latency during peak load. Remote fork, employing either Checkpoint/Restore (C/R) or Remote Direct Memory Access (RDMA), offers a potential solution for function scaling acceleration. Although RDMA fork is faster, the opportun...
System is presented, a reference-aware codec that retrieves exact-token historical spans for prefill uplinks, reuses the reconstructed uplink state for same-round prefill downlinks, and generates boundary-specific decode references with lightweight causal predictors.
This work presents AsymSpec, which addresses uplink-gated verification and invalid dependent work with two corresponding mechanisms, and shows that AsymSpec delivers 2.82-28.03$\times the output-token throughput of the strongest baseline.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
