NST-RL: Residual Reinforcement Learning With Approximate Optimal Regulation for Robust Drone Control

Abstract

We present NST-RL, a framework that couples residual reinforcement learning (RL) with an approximate optimal regulator computed using Nonlinear Systems Toolbox (NST) for robust control. Disturbance compensation is learned through residual corrections routed via a dynamic extension that preserves control-input positivity, attenuates high-frequency residual content, and keeps the nominal regulation locally input-to-state stable for bounded residuals. A BalancedBuffer uses the regulator’s optimal cost to concentrate off-policy training on high-effort states that a competent nominal controller would otherwise leave underrepresented in the data. We realize the framework for a drone trajectory tracking task with experiments on a Crazyflie 2.1 drone, where it shows improved tracking robustness and smoother control over geometric, adaptive, and prior residual RL baselines under wind and payload disturbances.