Skip to content

Author

Yi-Yu Liu

We have 1 of 4 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse

This work studies production traces from two companies and evaluates 14 eviction algorithms across HBM-constrained and large memory-pool settings to show that effective prefix-cache management should retain recency as its foundation while selectively adding quick demotion for one-hit prefixes.

Yi-Yu Liu, Min-Lan Yu, Jun-Cheng Yang · 3 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.