Skip to content
Open access

A Novel Tokenization-Free GPT-2 Mechanism for Intermittent Demand Forecasting

2026 · IEEE Access · Vol 14, pp. 137454-137474 · 0 citations · 51 references

TL;DR

A controlled comparative assessment within the GPT-2 family is run on 384 stratified M5 intermittent series, crossing two output-head architectures with two loss functions, showing the penalty rising monotonically with the average inter-demand interval.

Abstract

Intermittent demand — spare parts, slow-moving SKUs, long-tail retail — has been one of forecasting’s oldest open problems, and the recent wave of LLM-based forecasters has not yet settled the picture. Across the literature, the design choices that shape an LLM forecaster — backbone, output head, loss function, input representation, training scope — co-vary across studies, and the effect of any single choice on accuracy is rarely isolated. We run a controlled comparative assessment within the GPT-2 family on 384 stratified M5 intermittent series, crossing two output-head architectures (linear projection vs. two-layer MLP) with two loss functions (compound BCE+MSE vs. pure count NLL), holding the backbone, input representation, tuning protocol, and evaluation metric fixed. The head-architecture contrast is statistically indistinguishable from null under both losses; switching from compound BCE+MSE to pure count NLL increases per-series mean RMSSE by roughly 18% under both heads, with the penalty concentrated in the highest-intermittency strata where non-zero observations are rarest. Benchmarked against LSTM, LightGBM, and six classical Croston-style methods, the compound-loss GPT-2 configurations sit within the Nemenyi critical difference of LSTM at the headline 28-day horizon and track it closely across the reported horizons. For per-series intermittent demand forecasting with pretrained transformers, the supervision signal matters far more than the architecture of the head sitting on top of it; the design budget should reflect that. An external validation on 100 synthetic series with controlled intermittency, under replication criteria fixed in advance, reproduces the loss-function effect at matching magnitude (+17%) and shows the penalty rising monotonically with the average inter-demand interval.

Read PDF

Similar papers

#machine learning Preprint Sep 2026

Downside-Controlled Online Forecast Combination under Delayed and Revised Outcomes

Post-hoc correction adjusts a forecaster that cannot be retrained, such as a foundation model, but a correction fitted where errors are stable can hurt where they shift. We aim for downside control: not much worse than the starting forecast. We combine the frozen forecaster, a static corrector and an online corrector o...

Minkyoung Kim, Hyunjung Byun, Yo-Han Lee et al. · 0 citations
Open access Aug 2026

A Reproducible, Leakage-Free Pipeline for Censored Demand Forecasting and Inventory Optimization on FreshRetailNet-50K

Accurate demand forecasting for perishable goods is complicated by demand censoring: when stockouts occur, observed sales understate true demand and bias both forecasts and replenishment decisions. We present a leakage-free, end-to-end pipeline of three stages: an inverse-Mills-ratio (IMR) recovery heuristic benchmarke...

Joseph Azar · 0 citations
Preprint Sep 2026

Deep Learning of Robust Market Making under Regime-Switching Order Flow

A deep reinforcement-learning market maker (RLMM) is developed - a Rainbow-style distributional DQN (C51) which is calibrated and tested in a zero-intelligence limit order book and finds that, in the stationary setting, RLMM outperforms GLFT across the entire observed risk-return frontier.

F. Moret, Fabrizio Lillo · 0 citations
Conference Open access Oct 2026

Leakage-Controlled Chronological Evaluation of Hybrid Option Pricing Models

Option-pricing comparisons can be biased when quote-derived inputs or later outcomes enter model selection. We compare analytical, learned, fixed-blend, and conditional-blend models in a chronology that ends tuning in 2019 and fixes the 2024 to 2025 confirmation before its outcomes are examined. SPY, QQQ, and IWM are a...

Yu-Yang Zhao · 0 citations
#artificial intelligence Preprint Sep 2026

State Transport Routing for Short-horizon Adaptation in Multi-horizon Photovoltaic Forecasting

The proposed state transport routing (STR), a lightweight adapter that refines the predictions of a frozen forecasting model, demonstrates the potential of structured state adaptation to improve short-term forecasting across different neural architectures without retraining the underlying models or altering their longe...

Yu-Qing Xu, Li-Guo Zhou, Ze-Hua Sun et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.