Composable Building Blocks for Resilient Asynchronous Code
TL;DR
This work shows how higher-order combinators solve higher-order problems of asynchronous calls to a network service, database, or language model uniformly, including timeouts, retries, rate limiting, caching, reentrant locking, and cancellation.
Abstract
Asynchronous calls to a network service, database, or language model must cope with transient errors, slow or missing responses, throttling, and atomicity violations. We show how higher-order combinators solve such problems uniformly, including timeouts, retries, rate limiting, caching, reentrant locking, and cancellation. Every combinator maps an async function to another of the same type, so they share a uniform \emph{shape} and compose by nesting into one expression that implements a program's whole resilience and concurrency policy, leaving its business logic untouched. The same design spans both of JavaScript's native async shapes, promise-returning and async-iterable-returning functions, with one vocabulary of concerns. Solutions exist across the ecosystem but are scattered over differently shaped libraries that are hard to combine. We present case studies where the combinators are used to harden real packages by adding missing resilience or concurrency control and replacing bespoke policy.