A momentum-based estimator with mini-batches is employed to track the function values of each level, which are subsequently used to construct momentum gradient estimators, and a batch-free method that incorporates a first-order approximation and a clipping technique for function value estimation is developed.
Wei Jiang, Rui Yan, Si-Fan Yang et al.· 0 citations
Agnostic federated learning (AFL) seeks a model that performs reliably across $m$ heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency by reducing the number of synchronization rounds via faster convergence and the communication cost per round via compression. We first prop...
Hao-Min Bai, Jun-Yan Sun, Si-Fan Yang et al.· 0 citations
In non-stationary online learning, dynamic regret has attracted increasing attention as a measure of how well an online learner performs against a time-varying comparator sequence. Despite considerable advances, attaining optimal bounds for strongly convex and exp-concave losses often involves intricate analysis. In th...
Yi-Bo Wang, Wen-Hao Yang, Si-Fan Yang et al.· 0 citations
This paper studies projection-free algorithms for stochastic constrained multi-level compositional optimization and proposes variance-reduced projection-free algorithms and establishes complexity guarantees under both the Frank-Wolfe gap and the gradient mapping criteria.
Wei Jiang, Si-Fan Yang, Wen-Hao Yang et al.· 0 citations
Traditional variance reduction methods (e.g., SPIDER, SARAH, STORM) have been extensively investigated for improving the convergence rates of stochastic optimization. These techniques typically maintain a sequence of estimators for a single function (or gradient) across iterations. However, what if we need to track mul...
Wei Jiang, Si-Fan Yang, Yi-Bo Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.