Phase transitions and approximations of mean squared error for state-space models with fractional differencing

Abstract

We study trend estimation in state-space models in which the trend has a fractional stochastic difference of order $d>0$ and the observation errors form a short-range-dependent stationary process. Using finite-sequence fractional summation and differencing operators, we analyze the penalized least-squares estimator obtained by shrinking the fractional differences of the trend. We derive asymptotic mean squared error (MSE) approximations for all $d>0$ and identify a sharp phase transition at $d=1/2$. When $d>1/2$, the estimator is consistent and its optimally balanced MSE has order $n^{-(2d-1)/(2d)}$. At the boundary $d=1/2$, we obtain a refined finite-sample approximation and show that the MSE decreases at the slower order $\log\log n/\log n$. When $0<d<1/2$, the MSE converges to an explicit positive limit, so consistent recovery of the trend is impossible under the considered scaling. We also describe a practical criterion for choosing the penalty parameter and differencing order, and numerical experiments illustrate the MSE approximations and the behavior of the selection procedure.