Shadix: A Distributed Message Broker Leveraging Shared Disaggregated CXL Memory

Abstract

Message brokers are a core component of many data processing systems. They distribute messages from a set of producers to a set of consumers. Because of their central role, they need to offer low latency and high throughput. One approach to improve message broker performance is to specialize the implementation for modern hardware technologies. Compute Express Link (CXL) is a new interconnect technology based on PCI Express for cache-coherent access to remote memory. Multiple servers can use CXL to interact with shared resources and communicate over shared memory. CXL communication has different performance characteristics compared to technologies commonly used for communication with message brokers, like TCP. We present Shadix, the first message broker that leverages CXL shared memory. We compare throughput and round-trip time of our message broker with Valkey, NATS, and Apache Pulsar, three state-of-the-art message brokers. Shadix offers competitive performance with single-digit μs latency and saturates the throughput of the CXL hardware.