A KL Certificate for Best-of-$N$ Reranking in Language-Model Inference

Abstract

Best-of-$N$ reranking draws independent candidates from a reference policy and selects the response maximal under a fixed, sample-independent strict total order on outcomes. The selected law may differ substantially from the reference in Kullback–Leibler divergence. Prior work introduced a bounded statistic depending only on the accepted response's reference mass and conjectured that its expectation upper-bounds this divergence. This letter proves the conjecture for every finite ordered distribution. The proof applies to the full positive cumulative-distribution-function power family, not only integer sample counts. It combines a strictly monotone binary gauge, a top-atom chain-rule recursion, and induction, and yields an exact nonnegative slack decomposition and a quantitative tightness bound. A beta-quantile representation identifies the universal divergence cap. We also treat reference-preserving reward ties, provide deterministic high-precision illustrations, derive clipped fixed-sample confidence bounds, and specify stable evaluation and exact probability-logging requirements.