This paper develops a critic-free policy iteration (PI) method for continuous-time linear zero-sum games. The central idea is to characterize the saddle-point policies directly in the joint policy space, rather than treating the quadratic value matrix as an iterative variable. A policy game Riccati equation (PGRE) is introduced whose unknowns are policy gains only. Its solutions are shown to be in one-to-one correspondence with the symmetric solutions of the game algebraic Riccati equation. Based on a stabilizing anchor, PI is performed directly in the actor space. The actor-space Jacobian is nonsingular at every stabilizing policy, and the resulting policy sequence coincides with that of simultaneous PI. For unknown dynamics, a data-driven algorithm uses a single batch of data and nullspace projection of endpoint increments to eliminate the value matrix, yielding an actor-only regression. A necessary and sufficient rank condition for unique policy recovery is established and shown to be iteration-invariant, demonstrating that critic identifiability is unnecessary. A power systems frequency-regulation example verifies convergence and policy recovery, while scalability tests demonstrate substantial reductions in computational and memory requirements.
This article investigates data-driven policy iteration (PI) for continuous-time linear systems without requiring an initially stabilizing policy. Standard infinite-horizon PI is not self-starting because its policy-evaluation step is well posed only when the feedback gain is stabilizing. However, verifying this propert...
The resulting characterization reveals how control-dependent noise, policy-evaluation sensitivity, and update ordering govern local error propagation, and yields explicit sufficient conditions for local linear convergence.
Karl Handwerker, Felix Thömmes, Lucas Günther et al.· 1 citation
The entropy-regularized optimistic multiplicative weights update (ER-OMWU), a complementary single-loop policy dynamic that updates players'policies symmetrically and returns an approximate Nash equilibria in the last iteration, is proposed.
A novel adaptive optimal controller design scheme for continuous-time nonlinear systems through neural network-based policy iteration through Carleman linearization, which allows the optimal feedback control to be derived from a state-dependent Riccati (SDR) equation rather than the Hamilton-Jacobi-Bellman equation.
Jian-Guo Zhao, Zhi-Jiang Gao, Chun-Yu Yang et al.· Neural Networks· 0 citations
It is shown that, even for the covariance steering problem with a broad class of commonly used state and control safety constraints, the synthesized Markovian policy almost surely produces the same control actions as the history-dependent policy and therefore the same state trajectories, cost, and moments.
This letter develops a time-parallel policy-gradient framework for discrete-time nonlinear control-affine systems and proves that, for any finite horizon T, the state solver recovers the exact trajectory from any initialization in at most T iterations.
An Duy Nguyen, Lei-Lei Cui· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.