Meta-Learning Framework for Architecture and Data-Driven Loss Function Selection in Univariate Time Series Forecasting
Abstract
The selection of loss functions in deep time series forecasting often defaults to standard metrics such as MSE or MAE. Previous studies have largely focused on average predictive accuracy using classic architectures, ignoring the interactions between optimization objectives, architectural inductive biases, and temporal dynamics. This study evaluates the multidimensional impact of eight loss functions across seven neural network architectures. Using a full factorial design on six datasets from diverse domains, it executes 1,008 cross-validation runs across 336 scenarios. Statistical tests show that loss functions have a marginal, non-significant effect on average accuracy, but a significant influence on the location and boundaries of the residual distribution (mean, minimum, and maximum residuals), without significantly changing its dispersion or shape. The study also identifies gradient divergence, which occurred only in specific MLP-based architectures when applying Symmetric Mean Absolute Percentage Error to highly intermittent, zero-inflated data. These findings are synthesized into a meta-learning framework. A Decision Tree classifier, trained on statistical and architectural meta-features, achieved 85.42% accuracy (Macro F1-Score of 0.77) in predicting optimal loss categories; this accuracy reflects agreement with the quality-score label rather than a direct gain in forecast accuracy. Gini-based feature importance shows that architectural properties are the most important predictors, with the top architectural indicator accounting for over 30% of the model's total Gini importance. The framework provides a data-driven protocol designed to help avoid known failure modes, such as sMAPE-induced divergence in MLP models on zero-inflated data, and to prioritize worst-case residual behavior. Because Quantile Upper dominates the test set (64.4%), Macro F1 exposes minority-class performance; Quantile Lower was never selected, and the learned rule is conditional on the 0.9/0.1 quality-score weighting.