Exploiting Dynamic Similarity for Direct Transfer of MPC-based Policies

An Efficient Differentiable Model Predictive Control for UAV Attitude Regulation

Data-driven learning optimization, which considers optimization as a means to perform end-to-end learning, is an emerging methodology used to solve large-scale learning and continuous control tasks. These methods provide mathematically tractable solutions with inherent interpretability, but their training can be inefficient due to the need to differentiate through the optimization problem and to solve gradient information in every training iteration. To address this issue, we propose a novel learning algorithm called Efficient Differentiable Model Predictive Control (EDMPC), which significantly improves computational efficiency by integrating an optimization-driven inference step within the backpropagation process. More concretely, we derive an analytical gradient of the trajectory from the optimal problem relative to the tunable parameter, enabling significant acceleration of the end-to-end learning procedure. This gradient can be solved by a standard optimizer in a recursive form, which then provides the direction for parameter updates. An unmanned aerial vehicle control task was conducted to demonstrate the effectiveness of EDMPC in controlling complex dynamic systems. The results indicate that the proposed EDMPC method exhibits superior efficiency in the learning process and accuracy in the control process.

Inverse optimal control of general nonlinear systems based on adaptive dynamic programming

Parallel Policy-Gradient Methods for Parameter Optimization of Nonlinear Feedback Controllers

Structured feedback controllers provide rigorous stability guarantees, but often require manual parameter tuning to achieve good closed-loop performance. Policy-gradient methods offer a systematic approach to parameter optimization; however, conventional gradient evaluation requires sequential forward state rollout and backward costate propagation. This letter develops a time-parallel policy-gradient framework for discrete-time nonlinear control-affine systems. We derive the policy-gradient expression where the state and costate rollouts required for policy-gradient evaluation are formulated as residual-minimization problems and solved using Gauss-Newton (GN) iterations with parallel associative scans. For closed-loop systems that are globally asymptotically stable and locally exponentially stable, we show that the residual-minimization problems satisfy a local Polyak-Lojasiewicz (PL) inequality and that the GN iterates converge locally at a quadratic rate. Moreover, the PL constant, the size of the convergence neighborhood, and the quadratic convergence bound are independent of the rollout horizon T. We also prove that, for any finite horizon T, the state solver recovers the exact trajectory from any initialization in at most T iterations. Finally, an inertia-wheel pendulum example with interconnection and damping assignment passivity-based control (IDA-PBC) demonstrates improved closed-loop performance and the computational benefits of the proposed parallel policy-gradient framework.

Data-Driven Control Methods for Linear Discrete-Time Singularly Perturbed Systems

This paper presents a synergistic control strategy for discrete-time singularly perturbed systems, where data-driven learning is seamlessly combined with LMI-based synthesis, thereby offering an effective new approach for controlling discrete-time singularly perturbed systems in complex engineering environments.

HypEMBER: Hypernetwork-based Ensemble for Robust Policy Learning of Parametrized Dynamical Systems

In this work we investigate reinforcement learning (RL) as a framework for the robust control of parametrized dynamical systems in presence of measurements and model uncertainties. High-dimensional state spaces, expensive numerical solvers, the partial knowledge of the governing equations, and the dependence on physical parameters that may be uncertain or difficult to estimate accurately, make the use of standard RL approaches computationally unfeasible. Indeed, lack of robustness and poor generalization across parameter variations are further amplified in presence of noisy or incomplete measurements, ultimately hampering control performance. To address these challenges, we introduce HypEMBER, a novel RL framework based on the combination of hypernetworks and ensemble learning. In the proposed approach, both the policy and value functions are represented through hypernetworks that generate the weights of the underlying models conditioned on the physical parameters of the system, thereby enabling parametric generalization across different dynamical regimes. In addition, an ensemble of policy and value approximators is employed to quantify epistemic uncertainty, leading to improved exploration strategies and enhanced robustness during and after training. The performance of the proposed framework is assessed on two representative parametrized control problems: (i) the one-dimensional Kuramoto-Sivashinsky equation and (ii) a particle-navigation task in a two-dimensional time-dependent gyre flow, focusing on robustness with respect to measurement noise and parameter misspecification. Numerical results demonstrate that HypEMBER consistently improves training stability and sample efficiency, while achieving superior robustness to uncertainties affecting both the system dynamics and the available observations, in comparison with state-of-the-art RL methods.

Engineering Applications of Artificial Intelligence

A systematic assessment framework is presented that compares four prominent DRL controllers with a classical control baseline across a diverse set of applied control problems, including non-minimum phase dynamics, flexible mechanical systems, nonlinear marine control, and aerial robotics, and clarifies the trade-offs between learning-based and conventional control.