We have 4 of 29 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
This work presents AsymFlow, a prefill-decode disaggregated serving system that runs prefill on the GPU and decode on the CPU, and implements on SGLang and evaluated on a CPU-GPU platform.
The core of LEO lies in introducing hierarchical caching to construct a hybrid-grained software pipeline, enabling efficient computation-communication overlap both within and across GPU kernels.
Driven by cost and privacy constraints, many organizations deploy large language model (LLM) inference on local CPU-GPU servers. For long-context requests, the key-value (KV) cache grows linearly with sequence length and can rapidly exhaust GPU memory, reducing both the maximum supported context and request concurrency...
DPIO is presented, a unified I/O processing stack designed to harmonize the collaboration between CPU and DPU in NVMeoF environments, achieving near-optimal system performance across diverse workloads.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
