Skip to content
Open access

Safe and adaptive control of non-stationary stochastic systems via Lyapunov-constrained distributional reinforcement learning

Aug 2026 · Scientific Reports · Vol 16 · 1 citation · 59 references
Medicine

TL;DR

Simulations in chaotic and hyperchaotic systems showcase LG-DRL-ER’s superior performance in achieving stable, adaptive control compared to existing methods, and establishes finite-time convergence guarantees with explicit bounds on the convergence time.

Abstract

Non-stationary environments pose significant challenges for reinforcement learning (RL), particularly in safety-critical applications like robotics and energy systems, where adaptability, stability, and robustness to uncertainty are essential. This paper introduces Lyapunov-Guided Distributional Reinforcement Learning with Entropy Regularization (LG-DRL-ER), a novel framework designed to address these challenges in non-stationary Markov Decision Processes (MDPs). LG-DRL-ER integrates three key components: distributional RL to model the full return distribution, capturing reward uncertainty; Lyapunov stability constraints to ensure safe convergence to a target state; and entropy regularization to promote adaptive exploration. We present a rigorous stability analysis, supported by lemmas and theorems, demonstrating convergence in probability under dynamic conditions. We further establish finite-time convergence guarantees with explicit bounds on the convergence time. An adaptive parameter update rule ensures robustness to changing dynamics, validated through theoretical guarantees and empirical evaluations. Simulations in chaotic and hyperchaotic systems showcase LG-DRL-ER’s superior performance in achieving stable, adaptive control compared to existing methods. This framework advances safe RL by bridging data-driven learning with control-theoretic guarantees, offering significant implications for autonomous systems and dynamic environments.

Read PDF

Similar papers

Open access Sep 2026

Reinforcement-Learning Robust Tracking Control of Discrete-Time Systems with Diagonal Scaling

This paper addresses a robust tracking problem for linear discrete-time systems by proposing a reinforcement learning (RL) control method based on a diagonal-scaling strategy, offering a solution tailored to the demands of enhanced reliability. To overcome a common limitation in policy-iteration-based RL design, namely...

Kan-Yang Jiang, Zheng Gao, Ye Zeng et al. · 0 citations
Open access 2026

Constrained Residual Reinforcement Learning Augmentation of MRAC for Actuator-Safe Nonlinear Tracking

Robust tracking under nonlinear uncertainty and actuator constraints remains challenging for model-reference adaptive control (MRAC). Deep reinforcement learning (RL) can compensate for nonlinear effects, but a standalone policy provides no inherent model-reference structure. This paper combines an MRAC backbone with a...

Abdelrehim A. Elembaby, B. Emran · 0 citations
Sep 2026

Safety-Aware Optimal Tracking Control of Stochastic Systems With Obstacle Avoidance via Adaptive Dynamic Programming.

Safe trajectory tracking for nonlinear stochastic systems operating in obstacle-cluttered environments remains a significant challenge, as random disturbances and obstacle-induced constraints can simultaneously degrade tracking accuracy and threaten system safety. To overcome this issue, this article develops a safety-...

Lu-Lu Zhang, Hua-Guang Zhang, Xiao-Hui Yue et al. · 0 citations
Preprint Sep 2026

SafePG: Safe and Globally Optimal Reinforcement Learning with Hard Constraints

An optimal and convergent model-free policy gradient (PG) reinforcement learning (RL) reinforcement learning framework for controlling nonlinear dynamical systems under hard safety constraints is presented and a model-free PG algorithm based on stochastic gradient ascent is developed.

Vipul K. Sharma, Wesley A. Suttle, S. Sivaranjani · 0 citations
Sep 2026

Event-triggered reinforcement learning-based safe control for stochastic systems subject to asymmetric input constraints and unknown dynamics.

This paper investigates the safe optimal control for input-constrained unknown stochastic systems via adaptive dynamic programming (ADP) and generalized fuzzy hyperbolic model (GFHM) and the Lyapunov method is utilized to guarantee the uniform ultimate boundedness of the system states and the weight estimation errors o...

Yu-Ling Liang, Feng-Lin Qin, Lei Liu et al. · 0 citations

Learning Provable Neural Network Observer for Uncertain Dynamical Systems

A novel two-stage training framework for provably stable neural network observers that decouples the optimization into a point-guided Lyapunov pre-training phase, which rapidly achieves high estimation accuracy and local stability over sampled states and an LMI fine-tuning phase that efficiently satisfies a strict glob...

Zhang-Yi Wang, Jia-Xu Liu, Song Chen et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.