Narwhal: Breaking the Local Boundary via Disaggregated Memory Scheduling for Alibaba AnalyticDB
TL;DR
This work presents Narwhal, an RDMA-co-designed analytical database, which enforces hardware-level multi-tenant isolation with per-tenant RDMA Protection Domains, quotas, and QoS and eliminates OOMs and protect metadata.
Abstract
Cloud OLAP workloads are bursty and memory-hungry. Per-machine DRAM caps and co-scaled CPU/memory provisioning are fragile. They lead to low utilization, slow autoscaling, and transient OOMs. Prediction-driven schedulers and bin-packing optimizers mitigate but struggle with rigid per-machine memory boundaries and forecasting errors, Existing RDMA or OS-level remote-memory approaches either assume single-tenant trusted clusters (lacking hardware isolation and QoS) or treat remote DRAM as opaque swap. We present Narwhal, an RDMA-co-designed analytical database. It jointly schedules local DRAM and a shared remote memory pool to trade consolidation against bounded remote borrowing. It enforces hardware-level multi-tenant isolation with per-tenant RDMA Protection Domains, quotas, and QoS. It uses a database-aware memory allocator with a three-tier cache and a pressure-threshold invariant to avoid OOMs and protect metadata. Integrated into a production engine, Narwhal eliminates OOMs, boosts memory overselling from 92.67% to 121.89% (+29.22) and node utilization from 78.16% to 96.63% (+18.47) , while limiting average remote memory borrowing to just 1.63%.