Energy Measurement of Compressed Deep Learning Models: A Literature Review
Abstract
Model compression — quantization, pruning, and knowledge distillation — is widely adopted on the premise that a smaller model costs less energy to run. This review examines the measurement layer beneath that premise and finds it less settled than its adoption suggests. Drawing on a purposively assembled corpus of 53 works (2019–2026), supplemented by a logged literature search that tested six specific absence claims, the review traces why compression's energy benefit is not guaranteed: software energy estimators and hardware counters disagree by margins comparable to the savings compression claims to deliver; floating-point operation counts correlate weakly with measured energy once memory bandwidth, batch size, and backend kernel realization are taken into account; and compression rankings established on one hardware platform do not reliably transfer to another. A controlled evaluation would need to hold the measurement instrument, workload, and protocol fixed while compression state and platform vary, and no study in the corpus does so. The review's central contribution is not the observation that compression is unpredictable, but an account of the mechanisms — memory-bound execution, batch-size regime, and backend realization — that explain when and why it fails, together with a research agenda built around three of the resulting gaps: measurement-tool validation on compressed models, the nominal-versus-realized compression gap, and cross-platform rank-order preservation under a matched protocol.
