Skip to content
Book Open access

RPCShield: Defending Microservices Against Cascading Failures

Sep 2026 · Proceedings of the ACM SIGOPS 32nd Symposium on Operating Systems Principles · 3 citations · ⚡ 1 influential · 53 references

TL;DR

RPCShield is a suite of program analyses for Go and Java that detects cascading-failure risks in microservice-based systems, and has detected over 3,800 cascading failure risks, of which service owners have already remediated more than 1,300.

Abstract

Microservice-based systems are tightly interdependent. A failure in one service can propagate to a dependent service, potentially bringing down critical, user-facing functionality. We have developed and deployed RPCShield, a suite of program analyses for Go and Java that detects such cascading-failure risks. RPCShield uses static program analysis to track error and exception propagation through service code. Two insights make this practical. First, error propagation needs to be only checked per service, so the analysis is intra-service and avoids inter-service analysis entirely. Second, deciding the property counterfactually, by asking whether an endpoint could have succeeded had a given call succeeded, attributes the failure to the specific responsible dependency rather than to every dependency whose error reaches the caller. We have deployed RPCShield at Uber, across a fleet of over 6K microservices. Since deployment, it has detected over 3,800 cascading failure risks, of which service owners have already remediated more than 1,300. Manual review indicates that RPCShield has a false positive rate of around 5%.

Read PDF

Similar papers

Preprint Sep 2026

Backstitch: Restoring Request Causality Across a Production Microservice Fleet

A major video platform runs on thousands of microservices, each request propagating a context so downstream work can be traced and governed. At handoffs outside instrumented paths, e.g., custom queues and callbacks, the payload continues but the context does not, and the request still succeeds under existing tests. Suc...

Zi-Yue Dang, Qiu-Yu Wu, Hao-Yun Xu et al. · 0 citations
Open access Aug 2026

Devevopment of a method for improving maintainability of microservice software based on centralized outgoing event management

A method developed for centralized management of output events, consisting of six components, provides complete isolation of business logic from event transport mechanisms through declarative routing rules and dependence inversion and is appropriate for microservices with more than two event transport mechanisms or mor...

V. Lakhai, M. Seniv · 0 citations
Preprint Aug 2026

ORCA: Observability-Grounded Program Repair for Microservice Incidents

Results show that operational telemetry can be transformed from diagnostic evidence into actionable repair context: paired telemetry supports repair-oriented localization, while repair graph agents convert localized code and configuration evidence into constrained patch-generation context for the LLM.

Yuan-Chen Gao, Yi-Fang Tian, Yi-Ran Li et al. · 1 citation
Review Aug 2026

Can MCP Clients Decide What to Do After Failure? A Result-Only Actionability Audit

What deterministic software can learn from a completed MCP failure result alone is studied; request arguments, schemas, discovery history, authen- tication state, transport metadata, host policy, and host policy, and host policy, and ap- plication state are outside that boundary.

Rishabh Mehan · 1 citation
#artificial intelligence Preprint Oct 2026

Where Did the Repair First Go Wrong? Localizing the Origins of Silent Failures in Agentic Vulnerability Repair

Security Awareness Gap Evaluation (SAGE), a trace-based method that combines an assessment of the security reasoning recorded at each turn with the reconstructed code history to identify the earliest turn at which a repair diverges from the task's security intent, is proposed.

Wen-Jie Bai, Muhammad Waseem, Z. Rasheed et al. · 0 citations
Preprint Aug 2026

Outcome Monitors: Recovery Affordances for Silent Tool Failures

Outcome Monitors, which detect violations of outcome contracts mined from task-disjoint traces or derived from public schemas, are introduced, which detect violations of outcome contracts mined from task-disjoint traces or derived from public schemas.

Sugam Panthi, Rabab Abdelfattah · 2 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.