On the Convergence of Semi-Gradient TD: A Mean Field Perspective
Under this condition, it is proved exponential convergence to a unique fixed point and its bias relative to the unregularized Bellman solution and it is shown that the same mechanism extends to entropy-regularized off-policy control through soft Q-learning.
Haruki Settai, Tadashi Kozuno, Shinji Ito
· 0 citations