LST-Sim: An Efficient Simulation Platform for Large-Scale Model Training

Abstract

Existing model training simulators fail to support heterogeneous GPU architectures and various intra-server high-speed interconnect protocols (e.g., NVLink, HCCS), while suffering from low efficiency in full-packet-level network simulation. In this paper, we present LST-Sim, an efficient simulator designed for large-scale model training, balancing simulation accuracy and runtime efficiency. First, the simulator takes into account the impacts of computational heterogeneity across different GPU models and inter-GPU interconnection bandwidth on training latency, and constructs a theoretical model. Furthermore, we propose a domain-based traffic management mechanism that integrates theoretical analysis with network simulation to handle different types of communication traffic, thereby improving simulation efficiency. The effectiveness of LST-Sim is well validated on the Ascend 910B training cluster. Its simulation results are in close agreement with real-world measurements. Meanwhile, the domain-based traffic management mechanism significantly reduces simulation runtime by approximately 75% while maintaining high fidelity.