Recursive moment updates for scalable Bayesian nonparametric regression

Abstract

Bayesian nonparametric regression methods have wide applications in statistics and machine learning, but inference can often be expensive or require approximations. This paper introduces a class of recursive moment function updates which are closed form, interpretable, expedient, and fully quantify Bayesian uncertainty through the martingale posterior. The proposed recursive updates can be used off-the-shelf and encompass challenging scenarios such as nonparametric heteroscedasticity, heavy-tailed or skewed noise, and importantly generalize to the multi-output case with missing outcomes without any significant loss in tractability or speed. We show frequentist consistency under weak assumptions in the aforementioned scenarios, and demonstrate empirically that our proposed method is orders of magnitude faster than competing Bayesian nonparametric approaches, with competitive or even superior prediction and estimation accuracy.