Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Quantization-aware training (QAT) leverages lower-precision arithmetic to reduce the cost of LLM deployment, but aggressive quantization degrades final model performance. A common remedy is mixed-precision training, in which high precision is assigned to some of the layers to maintain performance while keeping the cost...
Alexandra Volkova, Matin Ansaripour, Erik Schultheis et al.
· 0 citations