Muon has emerged as a strong optimizer for the matrix-valued parameters in large language model pretraining, approximately orthogonalizing its momentum with a few Newton-Schulz iterations. Existing theory either replaces this iteration with the exact polar factor it approximates, or treats its finite depth as an approx...
This study delves into large PFMs adaptation in the resource-constrained federated learning environment, and proposes an innovative framework, namely ADRAP, which alternates between large model distillation and resource-adaptive pruning, with guaranteed convergence.
Xiao Zhang, Yang-Yang Wang, Xing-Yu Sun et al.· IEEE Transactions on Pattern...· 0 citations
Zero thermal expansion (ZTE) materials provide exceptional dimensional stability under temperature fluctuations, making them indispensable for high-precision instrumentation and extreme environments. However, their natural scarcity, combined with the vast compositional space in multicomponent systems, renders tradition...
Song Yang, Jing-Han Li, Wan-Da Yang et al.· Advances in Materials· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.