Caches reduce latency and network traffic, and cache performance largely depends on the eviction policy. Eviction effectiveness is typically measured by byte and object miss ratios. Although learning-based eviction policies can reduce misses, their high computational overhead limits practical deployment. This work pres...
Qian Wang, Chang Xu, Wenbin Zhou et al.· ACM Transactions on Computer...· 0 citations
Xronos is proposed, a collaborative fine-tuning framework for heterogeneous CPU edge devices that uses TP as its execution backbone and combines lightweight profiling with heterogeneity-aware tensor partitioning to reduce the straggler bottleneck.
Wonmi Choi, Sunjae Park, Dohyeok Kwon et al.· 0 citations
GeoMesh is presented, a synchronous geo-distributed training framework for heterogeneous GPUs that balances per-worker workloads by assigning each GPU a suitable batch size and number of inner steps, so faster GPUs do more useful work instead of waiting.
Changyong Shin, Jaerim Park, Mi-Gyung Kang et al.· 0 citations
The latency with respect to the resource dynamics of processing multiple requests and tasks concurrently is characterized and new optimization opportunities that exploit the resource dynamics of tasks are demonstrated: CPU-aware tool admission and task-aware CPU allocation.
Wonmi Choi, Minuk Park, Zhixiong Niu et al.· 0 citations
OptiFlow is presented, among the first LLM-driven frameworks for automated design of high-performance collective communication algorithms, with key insight is a two-layer decomposition: the LLM generates compact data-movement intent expressed in a domain-specific language, while deterministic scheduling algorithms comp...
Fei Long, Ziyue Yang, Kaihui Gao et al.· Asia-Pacific Workshop on Net...· 1 citation
OasisKV is presented, a memory-centric LLM inference system design that alleviates HBM capacity pressure by decoupling full KV-cache storage from HBM during LLM decoding and observes that future important tokens can be predicted accurately in advance using lookahead tokens drafted by speculative decoding (SD).
Can Xiao, Sukmin Cho, J. We et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
