Five-Minute Rule for Data Analytics in the Cloud

Abstract

Traditionally, deploying database systems required acquiring and managing large, expensive servers on-premise. These servers feature a memory and storage hierarchy, including CPU caches, DRAM memory, HDDs and/or SSDs, and sometimes tape. For almost 40 years, Gray and Putzolu’s five-minute rule [4, 10, 12, 13] has helped guide system architects to the break-even point between memory caching and direct local storage access for these systems. The rise of cloud computing has significantly changed the way systems are deployed. Cloud services offer on-demand computing and storage, removing the need for companies to acquire and manage their own hardware. Furthermore, the cloud has introduced a new layer of the storage hierarchy for database systems: object stores . This durability and low cost have made object stores ubiquitous in modern cloud-native databases and data infrastructures [5, 6, 16]. We believe a rule of thumb similar to the five-minute rule is needed for object caches and storage for disaggregated cloud data system designs. However, it is not straightforward to adapt the established rules to the cloud as they presume fixed hardware, while, in the cloud, resources are dynamic and costs are determined by usage. One can argue that accessing object stores over the network incurs higher latency and lower bandwidth than directly attached storage devices, placing them at the bottom of the storage hierarchy. However, advances in cloud storage and networking have significantly impacted cost-effectiveness, latency variability, and dynamic workload optimization, challenging these traditional assumptions. For example, in non-storage-optimized AWS instances, network bandwidth often exceeds local storage read bandwidth, and writing to local storage devices tends to be half as fast as reading, while writing to S3 can match network bandwidth. This paper is a summary of our earlier work work [9] that proposes a cost model and new rules of thumb to help system designers determine when caches become cost-effective for analytical workloads in the cloud. While perhaps unsurprising, our analysis on AWS shows that caches are beneficial when a system makes (1) two requests per hour for latency-sensitive workloads, or (2) seven requests per second for non-latency-sensitive workloads. These results are consistent with and help explain the near ubiquity of object store caches in cloud analytics systems.