Cremes: Cost-Efficient and Reliable Microservice Execution on Spot Instances

PREEMPT-FaaS: Taming Orchestration Times in Latency-Sensitive Serverless Environments

PRISM: Predictive Runtime In-place Scaling and Model Selection for Edge Microservices

PRISM, a prediction-guided runtime framework that jointly selects model variants and CPU allocations for containerized edge microservices, and adapts each pipeline stage in place and minimizes predicted CPU-package energy under deadline, resource, and offline model-level Quality of Result constraints is presented.

SLO-Driven Horizontal Container Autoscaling

Modern web services are required to meet critical non-functional requirements, including availability, responsiveness, scalability, and reliability, which are formalized through Service Level Agreements (SLAs). SLAs define Service Level Objectives (SLOs), such as latency, throughput, and uptime, that ensure consistent service quality. Failing to meet these objectives can incur penalties and harm a provider's reputation. At the same time, over-provisioning resources leads to unnecessary costs and inefficient utilization. Autoscaling mechanisms address this by dynamically adjusting the number of service replicas according to demand. However, conventional approaches typically rely on low-level metrics, such as CPU or memory usage, which limit the ability to optimize both SLO compliance and infrastructure costs. This paper presents an enhanced SLO-driven autoscaling methodology for containerized workloads in Kubernetes clusters, integrating response time SLO targets into the autoscaling process. The proposed approach improves decision-making over traditional autoscaling by balancing service-level performance with operational efficiency. Experimental evaluation of a prototype demonstrates clear benefits compared to the default Kubernetes Horizontal Pod Autoscaler.

Leveraging Low-Parameter LLMs for Self-Healing in Kubernetes-Based Container Orchestration

The growing complexity of runtime management in containerized clusters makes manual fault debugging time-consuming and costly. Self-healing aims to automate fault detection and recovery. A recent approach is to use large language models (LLMs), but most existing systems rely on large proprietary models with high computational or service costs. This paper investigates the feasibility of using low-parameter, on-premise LLMs for self-healing of containerized applications in Kubernetes clusters. Our proposed prototype leverages prompt chaining by splitting failure management tasks into root cause analysis and auto-remediation prompts, where each prompt is augmented with in-context examples. We allow the LLM to iteratively refine its output based on cluster feedback until remediation execution and recovery are successful. Evaluated on a microservice benchmark system with 15 real-time faults, our prototype achieves 96.67% overall accuracy. Results show that prompt chaining and iterative refinement significantly improve recovery effectiveness, while in-context learning yields limited benefit.

CELLServe: An SLO-Aware and Cost Efficient LLMs Serving System for Serverless Computing Environments

CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.

Prediction-Robust Service Deployment with Capacity-Aware Edge Admission

This work proposes CAPSUM, a capacity-aware admission policy with an elastic specialization, CAPSUM-E, and implements an exact local offline dynamic program and compares against direct common-model baselines and documented source-derived adapters for EDP-A, OREO, and uEDC-L.