Reinforcement Learning-Based Control for Smooth and Accurate Circular Trajectory Tracking in Mobile Robots

Abstract

This paper investigates reinforcement learning-based control for smooth and accurate trajectory tracking of a nonholonomic mobile robot. Three reinforcement learning algorithms, Deep Q-Network (DQN), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC), are implemented and evaluated, with conventional PID and Sliding Mode Control (SMC) controllers included as comparative baselines. The robot is modeled using unicycle kinematics, and the tracking task is formulated through body-frame longitudinal, lateral, and heading errors. A reward function is designed to penalize tracking errors, radial deviation, and excessive control effort while encouraging forward progression. Simulations are conducted primarily on a circular reference trajectory, with a figure-eight trajectory included for additional comparison. The results show that all controllers can follow the reference trajectories, although their tracking accuracy and motion smoothness differ. DQN exhibits noticeable control switching, while PPO produces smoother actions but retains tracking deviations. Among the evaluated controllers, SAC achieves lower tracking errors and smaller control variations. These findings suggest that SAC is a promising approach for accurate and smooth trajectory tracking in mobile robots.