Feature Map Compression for Split Learning in Mobile Wireless Environment: Theory and Practice

Abstract

Split Learning (SL) enables collaborative model training by distributing computation between resource-constrained mobile edge devices and a central server. In this distributed framework, mobile edge devices compute the early layers of the model and transmit feature maps to the server. However, due to practical mobile wireless bandwidth limitations, these feature maps should be compressed before transmission. In this paper, we propose a lightweight Mask-Encoded Sparsification (MS) compressor that enhances Top-<inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="qu-ieq1-3696694.gif"/></alternatives></inline-formula> sparsification with a narrow-bit mask, significantly reducing compression error while improving computational and communication efficiency. Our theoretical analyses reveal that feature map compression generally induces estimator bias in the gradients. Furthermore, we establish the relationships between compression error and convergence, as well as between the choice of cutlayer and output error. The first relationship demonstrates that reducing compression error facilitates convergence, confirming the advantage of MS, whereas the second provides theoretical guidance for determining the cutlayer, indicating that earlier layers are more sensitive to compression errors. Extensive experiments on vision and language models demonstrate that our method outperforms existing compression techniques in terms of compression error and model performance, achieving up to a 21.38× speedup in communication efficiency under wireless environments.