Huitian Wang

Balanced Sparse Tree: A Scalable Network Topology for Large Language Models

This work proposes a novel topology named the Balanced Sparse Tree (BST), which is a topology characterized by symmetric design and sparse connections, motivated by hypergraph theory and Steiner Systems, and demonstrates the superiority of BST over the state-of-the-art in network scale, latency, bandwidth, and cost.