GraphReformer: Information-Bottleneck-Driven Implicit Structural Inference for Efficient Transformer-Based Time Series Forecasting

Abstract

Multivariate long-sequence time series forecasting (MLSTF) is critical in applications such as electricity consumption planning, where accurate prediction requires models to capture complex temporal dependencies over extended horizons while remaining computationally efficient. Current MLSTF methods are largely built upon Transformer architectures, which, despite their strong performance, suffer from quadratic complexity and high memory usage due to dense self-attention, limiting scalability for long input sequences. To address this issue, we propose GraphReformer, a novel forecasting architecture that replaces the encoder-side dense self-attention module with NRI-based implicit relational inference. GraphReformer models temporal dependencies through probabilistic relational graphs and incorporates a Variational Information Bottleneck (VIB) to preserve prediction-relevant information while suppressing redundancy and noise. Based on the inferred relational structure, the model further performs time-step aggregation to capture both local and long-range dependencies efficiently. Experiments on representative long-sequence forecasting benchmarks demonstrate that GraphReformer achieves competitive or superior forecasting accuracy, yielding an average MSE reduction of approximately 24% compared with strong baseline methods. Meanwhile, it substantially cuts computational and memory costs, demonstrating that relational inference is a promising alternative to dense self-attention for large-scale time series forecasting.