SPADE-DFL is developed, a primal--dual method that allows the number of local function-value updates between neighbor exchanges to grow with the computation budget while preserving the nonprivate convergence order.
Abstract
Reducing communication in derivative-free decentralized learning requires controlling the disagreement accumulated over multiple local updates. This paper develops SPADE-DFL, a primal--dual method that allows the number of local function-value updates between neighbor exchanges to grow with the computation budget while preserving the nonprivate convergence order. For smooth nonconvex objectives under uniform query-moment bounds, the prescribed nonprivate schedule achieves a time-averaged stationarity and consensus bound of $\mathcal{O}(T^{-1/3})$ using only $\Theta(T^{2/3})$ communication rounds, where $T$ is the number of local updates per client. For private training, the accumulated data-dependent increment is isolated from the graph correction, allowing one protected state per client and round to generate all outgoing messages. We prove client-level differential privacy for the full interactive transcript and quantify the resulting optimization error over a finite horizon. Experiments on four classification tasks show that SPADE-DFL achieves higher mean test accuracy than existing decentralized learning methods.
balanced fractional exchanges are introduced, which compress the policy mixture into a single fractional base while retaining the exchange information needed by the Poisson analysis, and lead to an polynomial time algorithm with the same regret guarantee.
This work proposes Decentralized Barrier Follow-the-Regularized-Leader (Dec-BFTRL), and evaluates each agent's played action against the average of all local objectives, with applications to online continuous diminishing-return (DR) submodular maximization.
Yi-Yang Lu, M. Pedramfar, Vaneet Aggarwal· 0 citations
A sharp contraction-per-scalar bound within the matching class and convergence guarantees for the core and sparse-momentum updates are derived for the core and sparse-momentum updates.
Sheng-Jun Zhang, Ting Liu, Heng Zhang et al.· 0 citations
We study adversarial online maximization of nonnegative, non-monotone DR-submodular functions over compact convex down-closed sets. A learner commits each action before observing its objective and competes with the best fixed action in hindsight. We prove a comparator-uniform first-order inequality that gives coefficie...
Cross-silo federated reinforcement learning (FRL) trains a shared policy across silos that must simultaneously respect a per-round uplink budget and on-device privacy. Although differential privacy (DP) and gradient compression each have mature solutions in supervised federated learning, naively stacking them in the po...
Jian-Tao Xu· IEEE Transactions on Communi...· 0 citations
Decentralized learning eliminates single-point-of-failure risks inherent in parameter-server architectures, but its practical deployment is bottlenecked by the communication cost of exchanging full-precision gradients over peer-to-peer links. Existing compression techniques—random quantization, sparsification, and fixe...
Ze-Hua Cheng, Wei Dai, Jia-Hao Sun· Proceedings of the Internati...· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026