A Simple Example of Bayesian Nonparametric Inconsistency

Abstract

I present a very simple example in which a full-support prior over distribution functions has an inconsistent posterior, which I believe has instructive value. The example is $Y_i \stackrel{\text{iid}}{\sim} P_0$ under the Bayesian hierarchical model $[Y_i \mid P] \stackrel{\text{iid}}{\sim} P$ and $P \sim \int \mbox{DP}(\alpha, H) \, \pi_\alpha(\alpha) \ d\alpha$ when $\pi_\alpha(\cdot)$ has exponential (or heavier) tails, where $\mbox{DP}(\alpha, H)$ denotes a Dirichlet process with concentration parameter $\alpha$ and mean $H(\cdot)$; on the other hand, consistency is obtained under a light-tailed prior. This unifies and generalizes a consistency result of Freedman and Diaconis (1983) with an inconsistency result described by Ferguson et al. (1992).