Transformer-Enhanced Actor-Critic Reinforcement Learning for Simulation-Based Autonomous Spot-Welding Control of Robotic Manipulators

Abstract

: This study presents a Transformer-Enhanced Actor-Critic Reinforcement Learning framework for simulation-based control of sequential robotic spot-welding tasks. The method couples a shared temporal encoder with deterministic actor-critic learning, auxiliary safety and process-quality outputs, and a multi-objective reward for position, force, smoothness, energy, time, collision, and safety objectives. In the reported simulations against Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed Deep Deterministic Policy Gradient (TD3), the proposed configuration reached an average reward of 1268.9, a task success rate of 98.8%, a positioning error of 0.96 mm, a contact-force Root Mean Squared Error (RMSE) of 24.3 N, and a simulation-derived process-quality proxy of 0.94. A controlled 2 × 2 ablation separates the effects of the temporal encoder and reward formulation; both factors improve the descriptive metrics, although their interaction is sub-additive for several outcomes. The available record does not contain raw multi-seed data, authenticated physical trials, complete implementation hyperparameters, or instrumented latency and memory logs. Accordingly, no inferential-significance, metallurgical weld-quality, real-time, or hardware-feasibility claim is made. The contribution is therefore a simulation-based sequence-aware control formulation whose physical deployment requires independently documented validation.