Gupta Iddhant

The Effects of Low-Bit Quantization on Confidence Calibration in Small Language Models

This study looks at how confidence patterns shift when cutting data precision to 4 bits using NF4, applied post-training on Phi-3.5-Mini-Instruct, a small-scale language model packing 3.8 billion parameters. Shrinking precision cuts down memory demands while speeding up output creation; however, what remains unclear is...