Skip to content
#edge computing Open access

CeeTrain: Fast Geo-Distributed Model Training Across Heterogeneous GPUs and Networks in Cloud-Edge-End Computing Environments

Dec 2026 · IEEE Transactions on Parallel and Distributed Systems · Vol 37, pp. 2759-2775 · 0 citations · 82 references

TL;DR

This work proposes CeeTrain, a geo-distributed model training system tailored for heterogeneous GPUs and networks in CEE environments, which adopts a communication-centric design philosophy to tackle challenges arising from slow and unstable inter-region networks.

Abstract

Most existing training systems focus on a single region. In contrast, we envision that cross-region training offers more flexible GPU resource allocation and yields significant potential. However, the hierarchical cluster topology and unstable networks in the cloud-edge-end (CEE) environment, a typical cross-region scenario, pose substantial challenges to building an efficient and autonomous model training system. We propose CeeTrain, a geo-distributed model training system tailored for heterogeneous GPUs and networks in CEE environments. CeeTrain adopts a communication-centric design philosophy to tackle challenges arising from slow and unstable inter-region networks. It begins with a heterogeneous device profiler that identifies and groups devices based on both network and compute characteristics. Leveraging device groups, CeeTrain implements compact, zero-bubble pipeline parallelism, automatically deriving optimal parallel strategies. To further adapt to runtime variability, CeeTrain integrates a dynamic environment adapter that reacts to network fluctuations. Extensive evaluations demonstrate that CeeTrain achieves 1.3-2.8× higher training throughput compared to widely used and SOTA training systems.

Read PDF

Similar papers

Preprint Oct 2026

GeoTrain: Energy-Aware Automatic Parallelism for Foundation Model Training on Heterogeneous Geo-Distributed Resources

Training foundation models at the frontier typically requires large-scale dedicated and centralized clusters with fast interconnects - infrastructure that is prohibitively expensive and generally accessible only to well-resourced organizations. Decentralized training over pooled, geographically distributed GPUs offers...

B. Hoang, Tian-Zhang He, Mohammad Goudarzi · 0 citations
Open access 2026

Witcher: Training Large Language Models With Heterogeneous Spot GPUs in Data Centers

As the size of large language models (LLMs) continues to grow, training these models typically relies on data centers equipped with high-performance GPUs. However, in modern data centers, acquiring large-scale homogeneous GPU resources often results in long queueing delays, and training with on-demand instances incurs...

Chen-Hao Wang, Jun Wu, Ying-Hao Yu · 0 citations
Open access Sep 2026

GeoMesh: Workload-Balanced and Sign-Compressed Geo-Distributed LLM Training

GeoMesh is presented, a synchronous geo-distributed training framework for heterogeneous GPUs that balances per-worker workloads by assigning each GPU a suitable batch size and number of inner steps, so faster GPUs do more useful work instead of waiting.

Chang-Yong Shin, Jaerim Park, Mi-Gyung Kang et al. · 0 citations
#machine learning Preprint Oct 2026

Democratizing LLM Training with the Edge: Challenges, Systems, and the Road Ahead

The growing computational power of modern GPU data centers has let a few industrial giants train and deploy large language models (LLMs) on massive amounts of data. Yet the sheer scale of these models and their training data makes centralized development increasingly hard to sustain, and at the same time shuts out smal...

Zi-Yue Luo, Jiaxuan Cai, C. Le Denmat et al. · 0 citations
Book Open access Sep 2026

Chameleon: Adaptive Bidirectional Pipeline Parallelism for Edge Collaborative Training

Chameleon is a synchronous and adaptive bidirectional pipeline framework that achieves high throughput in dynamic and heterogeneous edge systems, and incorporates a hardware-aware throughput model to capture pipeline stage performance on heterogeneous devices.

Tao Liu, Xiang-Peng Hou, Li-Ying Li et al. · 0 citations
Open access 2026

Canvas+: Scalable Collective Communication Scheduling for Large-Scale GPU Clusters

State-of-the-art deep learning models rely on large GPU clusters and various parallelism strategies, which in turn depend on collective communication (CC) operators to synchronize data. While vendor libraries (e.g., NCCL, RCCL) provide standard CC algorithms, they often suffer from bandwidth bottlenecks in modern, comp...

Chen-Yang Hei, Fu-Liang Li, Dong-Lin Zheng et al. · 0 citations

Related blog posts

Microsoft Research Blog Oct 29, 2026

Microsoft Research Focus – Edition #97

Host Sowbhagya Suresh shares some of the latest news and cutting-edge research from across Microsoft—from understanding the challenges facing coding agents to uncovering how people use AI at work and improving extended-range weather forecasting. The post Microsoft Research Focus – Edition #97 appeared first on Microsoft Research.

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

Microsoft Research Blog Oct 21, 2026

Solving a 31-year-old grand challenge in mathematics – the KLS conjecture

For decades, a fundamental question about the difficulty of computing in high-dimensional spaces has resisted proof. New research resolves this question, removing a major theoretical obstacle to understanding the efficiency of randomized algorithms used across science and computing. The Kannan–Lovász–Simonovits (KLS) conjecture was proposed in 1995. It asks whether, for a broad class of high-dimensional […] The post Solving a 31-year-old grand challenge in mathematics – the KLS conjecture appear…

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.