Taming Update Drift in Asynchronous Federated Learning via Orthogonal Calibration
TL;DR
OrthoFL is proposed, an orthogonal calibration framework that decouples global and local learning progress to reduce interference and consistently outperforms state-of-the-art asynchronous baselines under various delay patterns and heterogeneity scenarios.
Abstract
Asynchronous federated learning improves the efficiency of conventional synchronous protocols by integrating updates as they arrive. However, asynchrony and data heterogeneity make learning objectives at global and local levels inherently inconsistent—global optimization trajectories can conflict with ongoing local updates. Existing methods simply distribute the latest global weights to clients, which may overwrite local progress and cause model drift. We propose OrthoFL, an orthogonal calibration framework that decouples global and local learning progress to reduce interference. Upon receiving an update, the server aggregates it into the global weights via an adaptive moving average. For local training, OrthoFL computes the global shift due to updates from other clients during the client's delay and removes its projection onto the received update. The resulting parameters lie in a subspace orthogonal to the client update and preserve the closest component of global progress within the orthogonal hyperplane. The calibrated shift is then merged into the client model for further training. Extensive experiments across five datasets demonstrate that OrthoFL improves accuracy by 7.4 percentage points and achieves up to a 12× speedup over the best-performing synchronous baseline. Moreover, it consistently outperforms state-of-the-art asynchronous baselines under various delay patterns and heterogeneity scenarios.