ResilienceBench-Operator: A Kubernetes Extension for Systematic Evaluation of Microservice Resilience Patterns
Microservice-based applications commonly employ resilience mechanisms such as Retry and Circuit Breaker to mitigate failures in service-to-service communication, yet configuring these mechanisms involves complex trade-offs between reliability and latency. Systematically evaluating these trade-offs in cloud-native Kubernetes environments remains a challenge due to the lack of tools supporting controlled, reproducible resilience experimentation. This paper presents ResilienceBench-Operator, a Kubernetes-native extension of the original ResilienceBench framework that enables in situ benchmarking of resilience mechanisms directly on microservice applications deployed in Kubernetes. The operator preserves the declarative evaluation-space model of ResilienceBench while integrating scenario generation, environment reconfiguration, fault injection, workload execution, and metric collection into Kubernetes through Custom Resource Definitions, controllers, and jobs. The paper formalizes the underlying scenario-based evaluation model, details the design and implementation of the operator, and demonstrates its capabilities through a controlled benchmark experiment on the Online Boutique application, covering hundreds of automatically orchestrated evaluation scenarios across multiple workloads, failure levels, and retry configurations. The experimental results reveal nuanced performance--reliability trade-offs and show, through Pareto-frontier and correlation-based analyses, how retry parameters affect checkout success and tail latency under different downstream failure and workload conditions. The paper also discusses practical challenges encountered when conducting controlled experiments in Kubernetes and extracts lessons that can guide future benchmarking efforts in this area.