PacketExpress: Fully Exploiting Large MTUs for Internet Traffic in Private Networks
Network bandwidth continues to scale rapidly, yet Internet data transmission performance remains constrained by the legacy 1500 B MTU. This small MTU translates high bandwidth into high packet rates that strain CPU processing at middleboxes and end hosts. While increasing the MTU could substantially improve performance, coordinating upgrades across arbitrary Internet paths is impractical. This paper presents PacketExpress, a packet processing architecture that enables networks to leverage large MTUs for Internet traffic without requiring modifications to neighboring networks. MTU-translating gateways at network borders dynamically aggregate incoming small packets into larger packets for efficient processing, then segment them back when forwarding externally. We present PXIO, a packet processing stack that leverages NIC offload capabilities to achieve high throughput. We introduce F-PMTUD, which determines the path MTU within a single round-trip time without relying on ICMP. For UDP, we present PX-caravan, a tunneling mechanism that encapsulates multiple packets to benefit from large MTUs while preserving packet boundaries. Our prototype achieves 1.47 Tbps throughput with 8 CPU cores while converting over 90% of 1500 B packets into 9000 B packets, improving middlebox performance by up to 5.1× and end-host performance by up to 2.5×.