It is argued that numerical format design is itself a first-class lever for stable FP4 training, and the first systematic study of FP4 LLM pretraining on energy-efficient Huawei Ascend NPUs is presented.
This work proposes a low-precision training framework based on 2D block FP4 quantization, which enforces transposition-invariant scaling and preserves consistency between forward and backward computations, and combines this with truncation-free scaling and stochastic rounding to control quantization error and maintain...
Mehdi Rahimifar, Amin Darabi, Mehran Taghian Jazi et al.· arXiv.org· 2 citations
This work proposes Residual Fallback Quantization (RFQ), a lightweight activation reconstruction framework that supplements the primary ulta-low-bit activation representation with an auxiliary quantized residual pathway that improves activation fidelity while preserving the efficiency advantages of ultra-low-bit comput...
Effective scaling of blockchain-enabled Industrial Internet of Things (IIoT) requires sharding that simultaneously ensures transaction locality, strict committee-size feasibility, and robustness against malicious node concentration. Existing methods often fail to balance this trilemma, risking either infeasible deploym...
Guangxia Xu, Zhuo Ye, Lu Wang et al.· IEEE Transactions on Network...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.