MLPFormer provides a practical hybrid artificial intelligence design for forecasting tasks in which trend and residual structures coexist, and confirms that the gains arise from matching encoder capacity to component complexity rather than from increasing parameter count.
Abstract
Long-term time series forecasting requires models that can represent smooth global trends and irregular local fluctuations without assigning the same capacity to both. Existing decomposition-based neural forecasters often separate signals but continue to process the resulting components with architecturally homogeneous encoders, which can overfit simple trends and underfit nonlinear residuals. This paper proposes MLPFormer, a hybrid MLP–Transformer framework for component-specific forecasting. The input sequence is decomposed by a moving-average filter into trend and residual terms. A shallow channel-wise MLP encodes the low-frequency trend, providing a low-capacity inductive bias for smooth structures, whereas a Transformer encoder with depthwise separable convolution models high-frequency residual dynamics and cross-variable dependencies. The two paths are fused and decoded by parallel linear and nonlinear prediction heads. Experiments on nine public benchmarks, with results reported as the mean over three random seeds, show average reductions of 11.8% in MSE and 8.1% in MAE relative to the mean of seven competitive baselines spanning Transformer-based, linear/MLP-based, and RNN-based paradigms published between 2023 and 2025. Ablation studies confirm that the gains arise from matching encoder capacity to component complexity rather than from increasing parameter count. MLPFormer provides a practical hybrid artificial intelligence design for forecasting tasks in which trend and residual structures coexist.
The Decomposed Recurrent Neural Network (DeRNN) is proposed, which decouples global trend modeling from local fluctuation extraction via an asymmetric dual-track architecture and exhibits superior robustness against noise and distribution shifts.
Shanyun Qian· Poster Volume 0008 The 2026...· 0 citations
Forecasting time series over long horizons is essential for proactive decision-making in many systems. Recent research has focused on transformer-based architectures, which capture long-range dependencies in sequential data. However, several studies show that simpler linear models can outperform transformers by avoiding overfitting during training. In this context, we present NeuroFlexMLP, a deep learning model for multivariate time series forecasting tasks. NeuroFlexMLP's key distinct feature is the adaptability to the diverse complexity of real-world time series, which is achieved, from the architecture standpoint, by adding non-linear residual blocks to a first linear block. This architectural design simplifies hyperparameter optimization, leading to accurate forecasts for various time series data types regardless of the lookback or prediction horizons, outperforming state-of-the-art (SOTA) models on challenging real-world datasets. Its Multi-Layer Perceptron (MLP) design ensures high computational efficiency, making it scalable for longer input sequences than transformer-based models. We validate NeuroFlexMLP for the LEO satellite beam hopping use case, where its lightweight design enables on-board deployment, and on state-of-the art AI datasets. Across all these benchmarks, NeuroFlexMLP achieves competitive accuracy over state-of-the-art models while providing an adaptive architecture that significantly reduces computational overhead. On the LEO beam hopping task, it achieves up to 35.9% MSE reduction over Informer, which translates into up to 28% lower provisioning cost under asymmetric cost models that penalize under-allocation more heavily than over-allocation.
P. F. Pérez, Claudio Fiandrino, Marco Fiore et al.· La Main· 0 citations
This paper proposes a time-series data processing model based on the Transformer Encoder, designed to address the limitations of traditional Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTM) in handling long sequences, such as long-term dependency issues, vanishing or exploding gradients, and low computational efficiency. By employing a self-attention mechanism along with learnable positional encoding and multi-head self-attention, the model effectively captures global dependencies in sequential data and significantly enhances its expressive power. To improve training stability, layer normalization is applied after the Transformer encoder, and the data processing flow is optimized through reorganization across batch and node dimensions. The model demonstrates clear advantages in parallel computation and long-term dependency modeling, enabling more efficient processing of complex time-series data. It exhibits superior computational efficiency and stronger generalization capabilities, particularly in real-time traffic flow prediction tasks based on large-scale datasets and long sequences. Experimental results show that the proposed model outperforms traditional RNN and LSTM models across various tasks, indicating promising potential for practical applications.
Ze Zhao, Mingyan Jiang, Feng Wang· International Conference on...· 0 citations
Forecasting financial indices remains difficult because market observations combine persistent movements, short-lived disturbances, nonlinear interactions, and substantial noise. A single recurrent architecture may retain nearby temporal information yet fail to represent widely separated dependencies, whereas an attention-based encoder is effective at relating distant positions but does not by itself guarantee detailed sensitivity to local sequence dynamics. This paper reformulates the forecasting pipeline as a cascaded global–local learning problem. A Transformer encoder first converts normalized price windows into contextual representations through positional encoding, multi-head self-attention, residual normalization, and a feedforward sublayer. Those representations are then processed by a bidirectional long short-term memory network, so forward and reverse recurrent states refine the global context before a dense prediction head produces the output. The design was assessed on daily closing-price series for the S&P 500, Dow Jones Industrial Average, and Nasdaq Composite over 2 September 2003–13 July 2023. Preprocessing included interpolation of missing observations, interquartile-range screening of outliers, first differencing, min–max scaling, and windowed sample construction. Tests against recurrent, bidirectional recurrent, feedforward, Informer, and temporal-convolution baselines show that the hybrid system delivers the strongest overall error and goodness-of-fit profile across the three markets. The findings indicate that passing attention-derived context into a bidirectional memory module offers a practical means of combining long-horizon structure with local temporal variation, although computational cost remains relevant for latency-sensitive trading applications.
Time series forecasting (TSF) is vital to many applications, yet existing models often struggle to capture the heterogeneous long-range global patterns and short-range local variations in multivariate time series. While some approaches partially model these dependencies, they often do not jointly exploit temporal and feature-wise information. To address this challenge, we propose HyBDM, a multi-scale hybrid model that decomposes temporal dynamics into global patterns and local variations, which are modeled by two specialized experts. The Global Patterns Expert employs an enhanced BiConv-Mamba module that integrates bidirectional convolutions, an M-SSM layer, a forgetting mechanism, and a GDD-MLP module for cross-channel modeling. The Local Variations Expert uses a Local Window Transformer (LWT) to perform efficient locality-aware attention with reduced computational complexity. In addition, a Multi-Scale Patcher and a Long-Short Router enable multi-resolution representations and adaptive fusion of the two experts. Experiments on six benchmark datasets show that HyBDM outperforms state-of-the-art methods in both forecasting accuracy and computational efficiency, demonstrating its effectiveness in bridging global-local dependencies for multivariate TSF.
Multivariate weather time series forecasting is highly challenging due to the complex coexistence of short-term dynamic fluctuations and long-term periodic patterns. Existing models predominantly focus on the time domain, lacking explicit mechanisms to capture frequency priors, or rely on complex spatial reconstructions that struggle with long-range temporal dependencies. To address these limitations, we propose the Time-Frequency Fusion Transformer (TFF-Former), a novel dual-branch architecture. Specifically, the time-domain branch utilizes a Transformer encoder to extract local dynamics and variable dependencies, while the frequency-domain branch employs the real Fast Fourier Transform (rFFT) combined with a parallel Transformer encoder to explicitly model multi-scale periodic structures from amplitude spectra. The representations from both domains are subsequently concatenated to achieve feature complementarity. Experiments on the Jena Climate dataset demonstrate that TFF-Former achieves state-of-the-art performance, outperforming strong baselines including TimesNet and PatchTST across MAE, MSE, RMSE, and SMAPE metrics. Ablation studies further validate that the time-frequency fusion mechanism significantly surpasses single-domain modeling.
Zhixiang Zhang· Applied and Computational En...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
