CoCoA: An in-CXL Switch Collective Communication Accelerator

Abstract

The Message Passing Interface (MPI) is widely adopted in high-performance computing (HPC) systems to support scalable inter-process communication. While HPC systems are theoretically expected to offer high performance, they often fall short in practice due to the overhead from MPI. In particular, MPI collective communications including numerous inter-process communications become a major bottleneck in data-intensive workloads. On the other hand, limited memory capacity and bandwidth further exacerbate this overhead, as large-scale data transfers require substantial memory resources. Therefore, reducing the collective communication overhead requires not only improving communication efficiency but also alleviating underlying memory limitations. Compute Express Link (CXL), an emerging interconnect standard, efficiently increases memory capacity and bandwidth by enabling memory-semantic access to remote memory. However, when CXL is deployed in large-scale systems such as HPC systems, it incurs additional latency (e.g., switching latency), potentially degrading collective communication performance. To mitigate the overhead of collective communication in CXL-based HPC systems, we propose an in-CXL switch Collective Communication Accelerator (CoCoA). CoCoA performs on-the-fly collective operations while retrieving data from CXL memory, thereby significantly reducing collective communication overhead. Experimental results show that the CoCoA-based system improves the performance of collective operations by 70.9% and 33.3%, on average, compared to the InfiniBand- and CXL 3.0-based systems, respectively.