GeoMesh: Workload-Balanced and Sign-Compressed Geo-Distributed LLM Training
GeoMesh is presented, a synchronous geo-distributed training framework for heterogeneous GPUs that balances per-worker workloads by assigning each GPU a suitable batch size and number of inner steps, so faster GPUs do more useful work instead of waiting.