CELLServe: An SLO-Aware and Cost Efficient LLMs Serving System for Serverless Computing Environments
CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.