Local String Stability-Driven Reinforcement Learning for Adaptive Swarm Coordination

Abstract

This paper presents a framework that integrates input-to-state string stability (ISSS) theory with reinforcement learning to achieve stable, scalable, and adaptive swarm coordination. First, a virtual string perception decomposes $N$-agent interactions into parallel robot-following subtasks by synthesizing local virtual neighbors through safety-weighted aggregation of sensed positions. Second, an actor learns adaptive proportional-derivative gain vectors from local observations of the virtual formation structure, enabling real-time gain adaptation without global knowledge. Third, an ISSS-driven reward function aligns the reinforcement learning objective with Lyapunov decrease conditions via potential-based reward shaping, allowing standard learning algorithms to discover stabilizing policies. Simulation experiments demonstrate sublinear convergence scaling and superior disturbance amortization compared to baselines; real-robot validation on Mona robots confirms stable formation coordination.