An Adaptive PID-SAC Algorithm for Robotic Constant Force Tracking of Massage Robotic Arm

Abstract

During continuous operation on the human back, the actual contact force exerted by a massage robotic arm may deviate from the desired value because of soft-tissue viscoelasticity, body-surface curvature changes, respiratory motion, and random disturbances. To achieve stable constant-force tracking, we propose a coordinated method combining an Extended State Observer (ESO)-enhanced proportional–integral–derivative (PID) controller with a temporally enhanced Soft Actor–Critic (SAC) algorithm to address response lag and high-frequency oscillations under complex noise. The two-layer architecture integrates fast compensation and policy optimization. In the PID-based layer, the ESO estimates the contact-force error, its first derivative, and the total disturbance; these estimates are used to schedule the PID gains and shape the controller output, improving contact establishment and continuous tracking. In the optimization layer, an improved SAC network adds Q-value-guided attention and frequency-gating constraints to long short-term memory (LSTM)-based sequence encoding, improving the utilization of historical states and suppressing high-frequency oscillations. Validation is conducted through PyBullet simulations of dynamic massage environments and experiments across stiffness levels of 1000 N/m, 2000 N/m, and 3000 N/m and target forces of 5 N, 8 N, and 10 N, demonstrating a certain degree of robustness to parameter variations. Under all conditions, the force error remains within ±0.3 N during stable tracking.