Code and data for "Reinforcement learning-based PID for nonlinear temperature control with delayed feedback and measurement noise"

Abstract

Code, data and notebooks accompanying the manuscript "Reinforcement learning-based PID for nonlinear temperature control with delayed feedback and measurement noise" (Y. Sapazhanov, S. Kadyrov; submitted to Mathematical Models in Engineering, Extrica). The study compares six strategies for tuning PID and nonlinear PID controllers on a simulated lumped-capacitance thermal plant with a state-dependent heat-transfer coefficient, actuator saturation and lag, a 5 s transport dead time and Gaussian sensor noise: Ziegler-Nichols, Cohen-Coon, differential evolution, a Smith predictor, and reinforcement learning in an offline (fixed-gain) and an online (adaptive) formulation. Every method is tuned from three seeds and evaluated on ten noise realizations at six operating points, and pairwise differences are tested with Holm-adjusted comparisons. A stability analysis (Lemma 1, Proposition 1, Theorem 1) gives the delay margin of the loop linearized at the operating point and is applied to every tuned gain set. Contents: the plant and controller code (core_plant.py), the main study script (study_v2.py) and its Colab notebook, four side-experiment notebooks (Smith predictor, Kalman-PID baseline, per-step timing, generalization to unseen operating points), all result CSV files behind the paper's tables including every tuned gain triplet, the delay-margin check of all 48 fixed-gain tunings, the noise-sweep data, and the script that regenerates the stability tables and figure in about one minute. All seeds are fixed; the numbers in the paper reproduce to the reported digits. See README.md for file-by-file descriptions and run instructions. Funded by Narxoz University under internal R&D contract No. I-490 (26 December 2024). Code: MIT; data and figures: CC BY 4.0.