EcoCompute: LLM Inference Energy on NVIDIA RTX 4090 (Ada) — FP16/NF4/INT8, 0.5B–7B, direct NVML, with paired perplexity, INT8 repeatability and a two-card NF4 re-test
Direct NVML energy measurements of decode-phase LLM inference on NVIDIA GeForce RTX 4090 cards (Ada, 24 GB), for five open models from 0.5B to 7B parameters in FP16, NF4 and INT8, with a measured run-to-run coefficient of variation and a two-physical-card re-test of the 3B NF4 anchor. Version 4 adds a fourth session (2...