Rethinking Near-Data Processing Execution: Enabling Device Collaboration with CXL Peer-To-Peer Communication
TL;DR
NDP-CP's task handoff between peers reduces latency by 12-46% over prior systems relying on host-mediated coordination, while deferring memory operations in NDP-CP leads to a 21-41% latency reduction over modern caching mechanisms for NUMA memory management without explicit data replication.
Abstract
We introduce near-data processing with collaborative peers (NDP-CP), a novel approach to NDP workload execution in modern systems. NDP-CP leverages Compute Express Link (CXL) peer-to-peer (P2P) communication to enable compute and memory interactions across peer devices. This design addresses the limitation in many existing NDP systems, which assume each NDP device operates independently, relying on host-mediated coordination and data replication. NDP-CP introduces two separate strategies to leverage peer devices. One enables task handoff between peer devices for decentralized workload execution. The other models CXL-based NDP systems with P2P communication as a non-uniform memory access (NUMA) architecture and reduces remote memory access overheads by deferring operations on remote data. We implement NDP-CP across various workloads, including approximate nearest neighbor search (ANNS), graph processing, and sparse matrix multiplication (SpMM), demonstrating its efficacy. NDP-CP's task handoff between peers reduces latency by 12-46% over prior systems relying on host-mediated coordination, while deferring memory operations in NDP-CP leads to a 21-41% latency reduction over modern caching mechanisms for NUMA memory management without explicit data replication.