SETTer is introduced, a transformer-based model that addresses challenges of long-term multivariate time series forecasting by incorporating novel techniques for decoupled self-attention and hybrid masking and outperforms state-of-the-art models in 88% of the scenarios.
Abstract
Long-term multivariate time series plays a significant role in many application areas such as power systems, trading, etc. However, their accurate prediction is quite difficult for conventional forecasting methods as they often exhibit high dimensionality and complex relationships. Recent works show that transformer-based approaches are quite effective for long-term forecasting thanks to their attention mechanism. However, in the presence of complex high-dimensional inputs, they show evidence of oversmoothing, limited capacity, and opacity. To this end, this paper introduces SETTer, a transformer-based model that addresses these challenges by incorporating novel techniques for decoupled self-attention and hybrid masking. The proposed techniques enable SETTer to effectively capture the dominant short- and long-term patterns across the temporal and channel dimensions. In addition, we enrich the model layers with simple explainable structures that indicate the discriminative pattern of SETTer. We show that with a single-layer transformer architecture, SETTer can effectively model long-term dependencies in the presence of varying data complexities. Extensive experiments on real-word benchmark datasets for long-term multivariate time series forecasting demonstrate that SETTer outperforms state-of-the-art models in 88% of the scenarios.
A novel architecture that combines a graph neural network block with a Long Short-Term Memoryencoder-decoder block for end-to-end feature learning and forecasting in multivariate time series forecasting is proposed, resulting in accurate and reliable predictions.
Ali Sarabi, Arash Sarabi, G. Runger· IISE Annual Conference &...· 0 citations
This paper proposes a novel architecture to improve forecasting performance over long horizons: the Multi-Scale Transformer, which effectively captures both short-term fluctuations and long-term trends by extending the transformer by integrating multi-scale processing.
YongKyung Oh, Alex A. T. Bui· IISE Annual Conference &...· 0 citations
This work starts from the underlying feature representation mechanism to deeply deconstruct data representation strategies which include dimensionality reduction decomposition, time-frequency domain conversion and masked pre-training, to provide a panoramic reference for the theoretical breakthroughs and engineering de...
De-Liang Zhang, Bo He, Yu Gao· Journal of Computing and Ele...· 0 citations
The findings indicate that the proposed architecture successfully reconciles multi-scale feature extraction with lightweight dependency modeling, enhancing structural generalization and providing a scalable framework for real-time temporal analysis in complex industrial environments.
This work proposes an inference-only streaming autoregressive framework that replaces repeated full-context recomputation with a one-time context warmup and incremental decoding, enabling efficient long-history forecasting without retraining.
Xi-Yu Meng, Yuhan Wu, Can-Ran Xiao et al.· Proceedings of the Thirty-Fi...· 0 citations
Large-scale time-series forecasting underpins decision-making in energy systems, traffic operations, industrial monitoring and financial analytics. Recent recurrent, convolutional and Transformer-based models have improved predictive accuracy, yet three limitations remain: inefficient modelling of long input windows, i...
Ya-Zhi Tao· Discover Computing· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026