Decision-Calibrated Quantile Selection for Time-Series Foundation Models in Tropical Precipitation Forecasting
Abstract
Time-series foundation models output a full predictive distribution, yet most evaluations judge them only by the median, a poor fit for daily rainfall, whose right-skewed, zero-inflated distribution keeps the median near zero on many rain days. We evaluate decision-calibrated quantile selection, a retraining-free application of validation-based threshold selection that replaces the median with the quantile maximizing validation F1 for the induced rain/no-rain classifier, and a magnitude-bounded variant that admits only quantiles whose validation RMSE stays within 10% of the median’s. Both are evaluated on four zero-shot foundation models (Chronos-2, Moirai, TimesFM, Lag-Llama) over a five-year test period at 22 rain gauges in Perak, Malaysia, with reconstructed weather covariates supplied as past-only context; for TimesFM this requires extending covariates flat across the horizon, a departure from its intended use. Every result is compared station by station with an always-predict-rain rule. Calibration moves every model off its near-zero median, but only some models gain discrimination. Chronos-2 beats always-rain at all 22 stations at one day ahead (Peirce skill score 0.09–0.14) and TimesFM does so at most stations, whereas Moirai’s F1-optimal rule is equivalent to always-rain, and Lag-Llama discriminates only weakly at one day ahead. F1-only selection can raise RMSE by more than 100%; the bounded variant prevents this for every model, and results are stable for tolerances between 5% and 20%. A comparison with classical baselines, an LSTM and Time-LLM shows that RMSE here mainly rewards approximating the climatological mean, motivating separate treatment of classification and magnitude error.