Influence functions for Gaussian process hyperparameter estimation

Abstract

We derive and analyse the influence function of hyperparameter estimators in Gaussian process (GP) models. While GPs are widely used for regression and probabilistic modelling, their hyperparameters are typically learned by maximizing the marginal likelihood. We show that the resulting estimators have unbounded influence functions, implying that even a single outlying data point can exert arbitrarily large leverage on kernel parameters. Using both theoretical derivations and empirical validations, we demonstrate that this lack of robustness is a structural property of GP hyperparameter estimation rather than a numerical artefact.