Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures
Large language model (LLM) outputs are expected to be reproducible under greedy decoding, yet in practice the same model, prompt, and software stack produce different outputs on different GPUs. The root cause is floating-point non-associativity combined with hardware-dependent kernel selection. Inference frameworks sel...
L. Cooper, Shinnung Jeong, Hyeran Jeon et al.
· 0 citations