The interface is implemented with a lightweight representation-level pre-encoder gate that assigns sigmoid scores to representation units, and the usage penalty reduces average admission scores while keeping forecasting errors close to the unpenalized TimeXer setting.
Abstract
Covariate-rich time-series forecasting requires deciding how external variables enter the target forecasting path. Existing Transformer-based forecasters usually build a covariate representation and pass it to the encoder without an explicit admission stage. This paper studies pre-encoder covariate admission as an input-side interface that regulates that representation immediately before encoder processing. We implement the interface with a lightweight representation-level pre-encoder gate that assigns sigmoid scores to representation units, and we also study a usage-regularized variant that penalizes average admission. The interface is evaluated as a plug-in module for TimeXer, Inverted Transformer (iTransformer), and Patch Time Series Transformer (PatchTST) under a zero-extra-tuning protocol, where each gated model inherits the corresponding baseline configuration. Experiments on the Electricity Transformer Temperature minute-level (ETTm1 and ETTm2) datasets, Traffic, Energy, and influenza-like illness (ILI) include paired forecasting comparisons, gate-placement ablation, initialization ablation, controlled covariate-admission analysis, and a variance inflation factor (VIF)-informed permutation feature importance (PFI) diagnostic case study. In the tested settings, the gate is competitive with the corresponding baselines, and the usage penalty reduces average admission scores while keeping forecasting errors close to the unpenalized TimeXer setting.
Multivariate long-horizon forecasting is critical to electricity load scheduling and traffic flow management, and to financial risk control. Existing deterministic backbones output a single trajectory, masking heterogeneous prediction difficulty across horizons and channels and providing no localized reliability signal...
A forecasting architecture that combines exponential moving average decomposition, a modified long short-term memory (LSTM) unit, and a Transformer is developed to improve short-term load prediction.
Xi Wang, Ling-Yu Zhang· International Conference on...· 0 citations
The accurate prediction of electricity load is a critical component for smart-grid management, energy allocation, and grid stability. Traditional statistical methods and Convolutional Neural Networks (CNNs) often struggle with nonlinear relationships or long-range temporal dependencies, whereas Transformer architecture...
This paper proposes a novel architecture to improve forecasting performance over long horizons: the Multi-Scale Transformer, which effectively captures both short-term fluctuations and long-term trends by extending the transformer by integrating multi-scale processing.
YongKyung Oh, Alex A. T. Bui· IISE Annual Conference &...· 0 citations
Short-term load forecasting plays a vital role in day-ahead scheduling, demand response, and energy-storage dispatch in distribution networks. The common recurrent neural networks always present lagged response around the peak-valley transition points and abrupt load change points, and the traditional transformer alway...
Yue-Lin Zou· International Conference on...· 0 citations
Short-Term Load Forecasting (STLF) is essential for maintaining a stable, affordable, and sustainable electricity supply across residential, commercial, industrial, transportation, and public sectors. Deep learning and ensemble methods such as Long Short-Term Memory (LSTM) and Extreme Gradient Boosting (XGBoost) have e...
A. Sanfilippo, A. Boumaiza, F. Ahmad· IEEE Open Journal of Industr...· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026