Aug 2026· Proceedings of the ACM SIGCOMM 2026 Conference· 0 citations· 53 references
TL;DR
GeoOrchestra is a system that decouples resource filtering from fine-grained strategy search by abstracting compute nodes via computation and memory profiles while modeling WAN links as a virtual hard pipe, which employs hetero-aware pruning to filter invalid resource sets and a resource-driven search that exploits resource disparities to maximize efficiency.
Abstract
Scaling Large Language Models (LLMs) increasingly mandates the aggregation of geographically dispersed resources. However, cross-datacenter (Cross-DC) training faces significant hurdles due to extreme hardware heterogeneity and limited Wide-Area Network (WAN) bandwidth. Empirical measurements on a 2,000 km Cross-DC testbed reveal that the interplay of hardware mismatches, combinatorial search complexity, and shared network contention creates severe performance bottlenecks for heterogeneous training. To address these complexities, we propose GeoOrchestra, a system that decouples resource filtering from fine-grained strategy search. By abstracting compute nodes via computation and memory profiles while modeling WAN links as a virtual hard pipe, GeoOrchestra deconstructs the parallel strategy search problem into three cascading stages. Specifically, it employs hetero-aware pruning to filter invalid resource sets, followed by a resource-driven search that exploits resource disparities to maximize efficiency. Finally, the system orchestrates training traffic via dynamic time-slot slicing to enforce lower-bound bandwidth guarantees for individual jobs. Evaluations on our testbed demonstrate that GeoOrchestra achieves up to 1.6× to 1.8× throughput improvement over state-of-the-art baselines, while maintaining stable execution under real-world shared WAN contention.
Large Language Model (LLM) training is increasingly concentrated in homogeneous datacenters, while private data and underutilized GPUs across universities, laboratories, and edge sites remain difficult to use. This extended abstract presents preliminary results from a geo-distributed LLM training prototype that treats networking constraints as first-order design concerns. The prototype connects three heterogeneous GPU sites via cloud-hosted parameter servers, outbound-only gRPC streams, two-stage delta compression (INT8 quantization + Huffman coding, achieving up to 4× payload reduction), and fault-tolerant rejoin. In real deployments, GPT-2 Medium pretraining achieves stable loss reduction and reaches the target loss 15.2% faster in wall-clock time than the best tested baseline; Llama3-1B pretraining remains stable under larger communication pressure; and cross-site latency traces reveal site-dependent WAN spikes of up to 200s. These results motivate adaptive networking support for synchronization, compression, placement, telemetry, and recovery in geo-distributed LLM training.
Ziyue Luo, Jiaxuan Cai, Cedric Le Denmat et al.· Conference on Applications,...· 0 citations
Training large-scale AI models often outgrows a single data center, demanding sharded, multi-cluster, and decentralized training. However, the huge space of resource allocations makes exhaustive benchmarking and manual tuning impractical, while performance depends on tightly coupled factors like model size, GPU memory, batch size, bandwidth, and sharding strategy. We introduce ShardMeter, a lightweight analytical performance model that predicts the end-to-end runtime of transformer-based workloads across arbitrary sharded, distributed, and even decentralized training. Given a model's characteristics and a target hardware topology, ShardMeter estimates per-GPU and per-island throughput, training cost, total wall-clock time, and identifies performance bottlenecks. Our analysis reveals diminishing-return regimes as island size increases, quantifies transitions between compute- and communication-bound scaling, evaluates hyperparameter trade-offs, and models cost-throughput for large-scale decentralized training. ShardMeter exposes these insights to quickly explore the configuration space, choose near-optimal deployment plans, and avoid costly trial and error.
Tim Beringer, Patrick Diem, Felix Wolf et al.· 0 citations
The power demands of large-scale AI training increasingly exceed the capacity of any single data center, making geo-distributed training across power-constrained sites a practical necessity. Prior work optimizes such training mainly for time-to-accuracy using single-tier aggregation, where every site exchanges model updates directly with a central aggregator over the WAN each synchronization round, without accounting for the energy required to reach convergence. Single-tier aggregation is fundamentally energy-inefficient because synchronization barriers force faster sites to idle, full WAN updates dominate communication energy at scale, and fixed synchronization frequency keeps paying the same communication cost even when updates shrink late in training. To address these inefficiencies, we present PowerScale, a hierarchical aggregation system that exploits the latency hierarchy of wide-area networks. PowerScale organizes sites into regional clusters and applies a Sync-Async synchronization modality: sites synchronize frequently with a nearby cluster aggregator over fast local links, while cluster aggregators push pre-aggregated updates asynchronously to a global aggregator over the WAN. PowerScale forms clusters based on both network proximity and power availability, and uses an adaptive synchronization policy that reduces communication energy by adjusting how often clusters synchronize to training progress. This structure shortens synchronization barriers and replaces per-site WAN transmissions with fewer, pre-aggregated transmissions at a lower frequency. We evaluate PowerScale at 100-site scale in a Flower-based simulation environment. PowerScale matches or slightly improves time-to-accuracy compared with single-tier baselines while reducing energy consumption by up to 3.9x.
Talha Mehboob, Zheng Xu, Michael Zink et al.· 0 citations
Deploying Large Language Models (LLMs) over the edge-cloud continuum faces severe stability challenges due to the conflict between stochastic network topology and complex workflow dependencies. Existing schedulers, relying either on computationally prohibitive Graph Neural Networks (GNNs) or topology-agnostic heuristics, fail to reconcile this tension. To bridge these gaps, we propose STEM, a service-level and topology-aware orchestration framework that formulates distributed LLM serving as a workflow-aware routing problem over a monitored service overlay, in which heterogeneous service instances act as specialized experts. At the core of STEM lies the STAR-PPO algorithm, utilizing a lightweight graph-free perception mechanism. By leveraging Squeeze-and-Excitation attention, it extracts critical bottleneck features from raw telemetry with linear complexity, bypassing the scalability limits of message-passing paradigms. To further achieve Pareto-efficient trade-offs, we develop a Dynamic Weight Adaptation (DWA) mechanism that autonomously recalibrates optimization preferences based on entropy-regularized metric drift. Extensive experiments on real-world datasets spanning 2,000 nodes demonstrate that our framework significantly outperforms state-of-the-art baselines. Specifically, STAR-PPO reduces network transmission costs by 96.8% and improves comprehensive inference efficiency by 24.4%, while sustaining robust zero-shot generalization across regions, with average latency within $1.09\times $ of a target-domain-retrained reference under a strict cross-region protocol. Code and data are available at https://github.com/gymorsiback/STARPPO
Yan Gao, Shaoyuan Huang, Yonghui Ye et al.· IEEE Transactions on Cogniti...· 0 citations
This work builds Silverstone to orchestrate dynamically both compute and network in disaggregated RL, using a reconfigurable optical-electrical fabric called RFabric that achieves superior performance-cost efficiency at scale over static Fat-Tree networks.
Xin Tan, Yicheng Feng, Yu Zhou et al.· Conference on Applications,...· 0 citations
Modern collective communication libraries (CCLs) execute a collective communication task (CCT) by decomposing it into multiple sub-tasks, each mapped to a specific Virtual Topology (VT), which is an ordered graph of GPUs (e.g., a ring or a tree), to maximize parallelism and link utilization. As AI training scales to larger clusters, network anomalies (congestion and failures) are unavoidable, and a single straggling VT can delay the entire CCT. Existing solutions either rely on low-level transport-layer solutions which lacks a cross-sub-task perspective, or static CCL scheduling, failing to adapt to the dynamic and heterogeneous networks. We present PReCCL, a drop-in NCCL replacement that combines software inband telemetry with cross-VT workload reallocation. Unlike hardware-dependent telemetry (e.g., P4-INT), PReCCL implements in-band monitoring within the CCL. It precisely measures the stall counts of each VT, and piggybacks the telemetry meta-data on existing collective traffic. With this fine-grained visibility, PReCCL employs a dynamic cross-VT reallocation mechanism that shifts workloads from straggler VTs (suffering from congestion or faults) to healthy ones. Reallocation occurs only at CCT boundaries using an epoch-based deterministic protocol, avoiding cross-rank consistency races. We implement PReCCL in NCCL and evaluate it on a 32-GPU testbed, large-scale simulation, and a 1024-GPU production cluster. On the testbed, PReCCL improves CCT completion by up to 2.1X and end-to-end training by 1.21X, with low overhead above tuned message-size thresholds.
Zhiyong Chen, Kaihui Gao, Li Chen et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
