MiniCache: Reusable Program Caching with Small Model Interfaces for Efficient LLM Inference
MiniCache is a reusable program caching framework that transforms Program-of-Thought programs into parameterized cache objects, enabling reusable computation across structurally similar requests, and consistently improves the trade-off between inference latency, cache reuse, and task quality.