Efficient Distributed Shared Memory System with Recall-Based Hybrid and Dynamic Coherence

Abstract

Distributed Shared Memory (DSM) transparently provides a unified global address space across a cluster, simplifying the development and deployment of distributed applications. However, existing DSMs synchronize data at a static page, fixedblock, or object granularity, which cannot match the diverse and evolving spatial locality of workloads and leads to either high protocol costs or unnecessary data movement. This paper presents DCDSM, a DSM system with hybrid and dynamic coherence granularity. By matching each coherence unit to the observed access footprint, DCDSM prefetches necessary data while avoiding unnecessary synchronization. DCDSM employs an invalidation-aware density-based policy to adapt coherence units according to effective spatial density. A recall-based cache coherence protocol supports this adaptation by decoupling fine-grained metadata management from variablesized data transfers and resolving granularity mismatches across nodes. We evaluate DCDSM against GAM, a state-of-the-art static-granularity DSM, on diverse distributed hash table workloads, showing that DCDSM improves throughput by up to $\mathbf{2. 0 6} \times$ and reduces RDMA bandwidth by up to 77.2%.