Quantum-Enhanced Multi-Agent Reinforcement Learning for Ubiquitous LLM Inference via Embodied UAV Swarms

Abstract

6G mobile edge networks are emerging as a key infrastructure for ubiquitous large language model (LLM) inference services. However, conventional edge routing to nearby or well-connected servers falls short for efficient edge LLM inference, as it may miss the user’s KV cache and trigger costly prefill recomputation. To address this challenge, this paper studies an edge inference system assisted by an embodied UAV agent swarm, where UAVs actively sense user mobility and neighboring UAV states to make local decisions on trajectory control, user association, and inference-request routing. The goal is to improve KV-cache reuse while maintaining reliable wireless connectivity, thereby maximizing the system effective token throughput under energy and QoS constraints. We then formulate the joint optimization as a mixed-integer non-linear program and further cast the sequential UAV decision-making process as a decentralized partially observable Markov decision process. To obtain scalable decentralized policies under partial observations, we propose Q-MAA2C, a quantum-enhanced multi-agent advantage actor-critic algorithm for embodied UAV swarm control and inference routing. Q-MAA2C uses quantum actors for local action selection and an entangled split critic for swarm-level value estimation, enabling coordinated policies from partial observations with reduced raw observation exchange. Simulation results indicate that Q-MAA2C yields comparable reinforcement learning rewards to the fully classical baseline while reducing the number of convergence episodes by about 43%. Additionally, the proposed method enhances the system effective token throughput by up to about 134% over other competing methods.