Communication-Efficient Decentralized LLM Inference over Low-Bandwidth Distributed Nodes

AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inference

AceSpec, an asymmetric edge-cloud collaborative framework that employs an asymmetric communication protocol that transmits minimal main-chain indices uplink and compact sparse distributions downlink and introduces a network-aware, Lagrangian-optimized resource allocation strategy that dynamically maximizes the local cache hit rate.

Broadcast Rate Limits in Wi-Fi: A Forgotten Bottleneck for Collaborative Edge LLM Inference

LLM deployment is migrating from data centers to edge devices, where Mixture-of-Experts (MoE) models offer a promising path: sparse expert activation allows the model to be spread across multiple low-cost edge nodes. Distributed MoE inference repeatedly dispatches embeddings from one main node to many workers - a one-to-many pattern poorly served by the sequential unicasts of mainstream stacks (NCCL, TCP), yet naturally matched by UDP broadcast. We propose a UDP broadcast method for collaborative edge MoE inference, augmented with timeout-driven retransmission exploiting near deterministic latency in distributed MoE for reliability and unordered result gathering for robustness to expert mispredictions, yielding a consistent 1.4x speedup over NCCL and TCP on a wired 8-node cluster. In wireless settings, however, we uncover a deeper, long-forgotten bottleneck: IEEE 802.11 caps broadcast rates at 54 Mbps regardless of physical-layer capacity - a legacy policy built for sparse control traffic, not edge AI. NS-3 simulations at distances 1m, 2m and 5m show that the optimal rates are much higher (64x, 43x, and 32x, respectively) than the 54 Mbps cap applied in standard. Thus, we argue that broadcast is no longer a control-plane relic: it is time for Wi-Fi standards to treat it as a high-throughput data-plane citizen.

Large Language Models in Wireless Communications: Applications and Challenges

It is concluded that the co-design of lightweight, telecom-specific large language models (Telecom-LLMs) and distributed inference mechanisms constitutes a pivotal evolutionary pathway toward realizing endogenous intelligence in future wireless communication systems.

RAC: Reference-Aware Activation Compression for Communication-Efficient Split LLM Inference

System is presented, a reference-aware codec that retrieves exact-token historical spans for prefill uplinks, reuses the reconstructed uplink state for same-round prefill downlinks, and generates boundary-specific decode references with lightweight causal predictors.

ToCo-Split: Topology-Constrained Communication for Efficient Split Learning in Edge Intelligence

Toward WAN-Aware LLM Training Across Heterogeneous, Geo-Distributed Sites

Large Language Model (LLM) training is increasingly concentrated in homogeneous datacenters, while private data and underutilized GPUs across universities, laboratories, and edge sites remain difficult to use. This extended abstract presents preliminary results from a geo-distributed LLM training prototype that treats networking constraints as first-order design concerns. The prototype connects three heterogeneous GPU sites via cloud-hosted parameter servers, outbound-only gRPC streams, two-stage delta compression (INT8 quantization + Huffman coding, achieving up to 4× payload reduction), and fault-tolerant rejoin. In real deployments, GPT-2 Medium pretraining achieves stable loss reduction and reaches the target loss 15.2% faster in wall-clock time than the best tested baseline; Llama3-1B pretraining remains stable under larger communication pressure; and cross-site latency traces reveal site-dependent WAN spikes of up to 200s. These results motivate adaptive networking support for synchronization, compression, placement, telemetry, and recovery in geo-distributed LLM training.