Leveraging Low-Parameter LLMs for Self-Healing in Kubernetes-Based Container Orchestration
The growing complexity of runtime management in containerized clusters makes manual fault debugging time-consuming and costly. Self-healing aims to automate fault detection and recovery. A recent approach is to use large language models (LLMs), but most existing systems rely on large proprietary models with high computational or service costs. This paper investigates the feasibility of using low-parameter, on-premise LLMs for self-healing of containerized applications in Kubernetes clusters. Our proposed prototype leverages prompt chaining by splitting failure management tasks into root cause analysis and auto-remediation prompts, where each prompt is augmented with in-context examples. We allow the LLM to iteratively refine its output based on cluster feedback until remediation execution and recovery are successful. Evaluated on a microservice benchmark system with 15 real-time faults, our prototype achieves 96.67% overall accuracy. Results show that prompt chaining and iterative refinement significantly improve recovery effectiveness, while in-context learning yields limited benefit.