Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking

Abstract

This paper investigates the performance of Echo State Networks (ESNs) for univariate forecasting of monthly and quarterly time series from the M4 Forecasting Competition dataset. We evaluate whether a simple first-order autoregressive ESN can serve as a competitive alternative to widely used forecasting methods. The study uses a two-stage design: a Parameter dataset is used to analyze ESN model configurations over leakage rate, spectral radius, reservoir size, and regularization selection, while a disjoint Forecast dataset is reserved for out-of-sample benchmarking. Forecast accuracy is measured using mean absolute scaled error (MASE), symmetric mean absolute percentage error (sMAPE), and the overall weighted average (OWA). The ESN is compared with simple benchmarks, statistical models including autoregressive integrated moving average (ARIMA), exponential smoothing state space (ETS), the Theta method, and TBATS, as well as multilayer perceptron (MLP), recurrent neural network (RNN), and hybrid exponential smoothing-RNN (ES-RNN) benchmarks. The model-configuration analysis reveals frequency-specific patterns: monthly series tend to favor moderately persistent reservoirs, whereas quarterly series favor more contractive dynamics; across both frequencies, high leakage rates are generally preferred. In the final benchmark, the ESN performs on par with ARIMA and TBATS under mean MASE for monthly data and achieves the second-lowest mean MASE for quarterly data. ES-RNN provides the best aggregate accuracy for both frequencies. Overall, the results indicate that a simple autoregressive ESN can provide competitive forecast accuracy on the considered filtered M4 subsets, particularly under MASE, while requiring low training and forecasting time once the ESN configuration has been fixed.