Do Reinforcement Learning Agents Improve Commodity Sector Rotation? Walk-Forward Evidence from Expert Selection, Strong Benchmarks, and a Frozen-Policy Temporal Extension

Abstract

This paper tests whether reinforcement learning improves monthly commodity-sector rotation once the experiment is reconstructed from investable instruments, strong active benchmarks, realistic costs, and strictly chronological validation. Total returns for energy (DBE), gold (GLD), agriculture (DBA), and base metals (DBB) are obtained from CRSP; GSG and DBC are investable broad-commodity benchmarks. Twelve lagged market-state features generate month-t+1 decisions. The initial training sample contains 132 targets through December 2018, the original holdout contains 72 months through December 2024, and a frozen-policy temporal extension adds 17 months through May 2026. All active results deduct 10 basis points per unit of drift-adjusted turnover, and Sharpe ratios use contemporaneous Treasury-bill returns. Six-month momentum earns an 18.6% CAGR and 1.112 excess Sharpe; a fixed 10-seed PPO ensemble earns 9.6% and 0.438. An expanding supervised expert selector earns 16.0% and 0.755. In an explicitly exploratory memory-window sensitivity, a 60-month rolling selector reaches 23.4% and 1.033, but its mean advantage over momentum is not statistically established (p=0.323). The pattern is consistent with time variation, but it neither identifies an optimal window nor establishes that older observations are harmful. PPO’s temporal-extension surge is concentrated in March 2026 and reverses when that month is removed. Direct deep RL therefore does not robustly dominate; constrained expert selection remains a research candidate whose memory sensitivity requires prospective confirmation.