Advancing Quantization for R&R Models with Microscaling Formats
This paper presents an empirical evaluation of both MXFP8 and the 6-bit MXFP6 format for industrial-scale R&R inference on the AMD MI350X architecture and demonstrates that MXFP8 is a viable alternative to row-wise FP8, and is the first paper, across R&R and LLM domains, that reports significant speedups from native MX...