BITS, a standardized, reproducible, and extensible benchmark for advancing research on irregular time series forecasting, is proposed and it is found that method performance varies substantially across irregularity characteristics, with no single modeling strategy consistently dominating.
Abstract
Despite recent progress in irregular time series forecasting, the field still lacks a unified benchmark for fair and comprehensive evaluation. Existing evaluations are often conducted on a limited set of datasets with inconsistent experimental protocols and predominantly error-based metrics, rendering it difficult to compare and assess methods fairly and comprehensively across diverse settings. To eliminate these limitations and accelerate progress, we propose BITS, a standardized, reproducible, and extensible benchmark for advancing research on irregular time series forecasting. BITS covers eleven datasets from nine domains with diverse irregularity characteristics, and it characterizes the datasets according to their missing rate, missing pattern complexity, sampling irregularity, and skewness. Further, it offers a unified pipeline for data preprocessing, model integration and evaluation, and reporting. It accommodates regular and irregular time series forecasting methods, including time series foundation models, under consistent settings, incorporating both error-based and non-error-based evaluation metrics. Findings include that method performance varies substantially across irregularity characteristics, with no single modeling strategy consistently dominating. We also find that using error-based or non-error-based metrics can yield different model rankings, highlighting the need for multi-dimensional evaluation. The code can be found at https://anonymous.4open.science/r/BITS-8F2E/.
Large Language Models (LLMs) have recently been applied to time series forecasting via prompt-based inference, yet their evaluation remains limited by potential benchmark contamination and an overemphasis on accuracy. We introduce a systematic benchmark that evaluates LLMs across accuracy, efficiency, cost, and environ...
Zairo Bastos, Fernanda Scarcela, Wesley Barbosa et al.· International Conference on...· 0 citations
Forecasting irregularly sampled multivariate time series (IMTS) remains a fundamental challenge in contemporary time series analysis, owing to the inherent non-uniformity of sampling and pronounced inter-variable asynchrony. While recent advances in context learning achieves remarkable performance, microscopic signal d...
Qing-Hua Ren, Yan-Lin Wu, Da-Long Gao et al.· Journal of Supercomputing· 0 citations
Time series forecasting (TSF) is a fundamental task in diverse decision-making scenarios, such as business intelligence, energy management, healthcare, and scientific discovery. Over the past decades, TSF has been extensively studied, with methods evolving from traditional statistical models to recent data-driven deep...
Ming-Yue Cheng, Zhi-Ding Liu, Xiao-Yu Tao et al.· IEEE Transactions on Knowled...· 21 citations
While established literature underscores the pivotal role of preprocessing in forecasting accuracy, this stage remains largely overlooked in current research. Modern benchmarks typically resort to simple scaling, failing to account for critical transformations required to address nonstationarity, such as differencing....
G. Padilha, P. D. de Mattos, Rafael M. O. Cruz· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 13, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
MIT News · Artificial Intelligence· news.mit.eduSep 29, 2026
Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.