Ensembling Language Models with Sequential Monte Carlo
This work introduces a unified framework for composing multiple LMs into ensemble distributions parameterized by a broad family of aggregation functions, and introduces an SMC algorithm that operates in a shared byte space, enabling ensembles of models with mismatching vocabularies and consistent sampling in the limit.