ElasticScale: Elastic Large Language Model Reinforcement Learning Training on Heterogeneous Mobile Edge Clusters
ElasticScale is presented, an elastic orchestration system that organizes heterogeneous accelerators into disaggregated rollout and trainer instances, via a HeterogeneousRayWorkerGroup abstraction that manages non-uniform hardware topologies and a multi-instance Federated Weight Averaging protocol that aggregates updat...
Wei-An Lin, M. Reza, Talha Nayyar et al.
· 0 citations