Communication-Efficient Over-the-Air Federated Learning via Lightweight Gradient Compression

Abstract

Integrating over-the-air computations into the model aggregation process of federated learning (FL) offers a promising solution to mitigate the communication bottleneck in FL model training. In this approach, all the clients modulate their intermediate parameters, such as gradients, onto the same set of orthogonal waveforms and transmit the resulting signals to the edge server simultaneously. Capitalizing on the superposition property of the radio channel, the server can extract an automatically aggregated global gradient from the received radio signal. However, the limited number of orthogonal waveforms imposes a constraint on the dimensionality of transmittable updates, hindering the adoption of more advanced, but high-dimensional models. In light of this challenge, we propose OFLight, a lightweight, yet effective, gradient compression algorithm tailored for OTA-FL systems. Specifically, in each communication round, the edge server constructs a low-rank projection matrix based on the received gradient matrix from the previous round (initialized with an independent and identically distributed standard normal matrix in the first round) and broadcasts it, along with the global model, to all clients in the system. Based on this matrix, every client projects its locally updated gradient matrix into a low-dimensional subspace through a linear operation. The clients upload only their compressed gradients via OTA computations, and the edge server can perform a linear decompression on the received signal, retrieving the original gradient dimension. Moreover, an error feedback mechanism is incorporated to compensate for the approximation error under aggressive compression. We derive analytical expressions for the convergence rate of both convex and non-convex loss functions, quantitatively demonstrating the effect of OFLight on the OTA-FL training efficiency. We also conduct extensive experiments to corroborate the efficacy of the proposed method.