P. Basta

Towards Green AI: Evaluating Energy Efficiency for Quantized LLM Inference

While it is known that quantization of large language models (LLMs) reduces memory usage via lower-bit weights, studies quantifying the resulting impact on energy usage and carbon intensity are scarce. We present the first unified evaluation of weight-only quantization strategies for LLM inference across varying input/...