Large Language Models for UAV Autonomy from a Perception–Cognition–Action Perspective
Deployable autonomy remains a key challenge for unmanned aerial vehicles (UAVs) operating in open-ended missions. Large language models (LLMs) and their multimodal variants, which can process visual and other sensory inputs, have introduced new capabilities for semantic perception, task reasoning, and language-conditioned control. However, these capabilities do not by themselves produce flight-ready autonomy. We structure our analysis around a Perception–Cognition–Action (P–C–A) framework. At each layer, we identify the capabilities contributed by LLM-based components and examine how they connect to existing flight modules through input specifications, output representations, architectural coupling patterns, and safety mechanisms. Across the surveyed systems, LLMs extend UAV autonomy beyond fixed perception categories, scripted task plans, and pre-programmed controllers. However, field deployment depends on whether model outputs can be transformed into representations that downstream modules can parse, verify, and safely execute. Without adequate validation, captions, task plans, code, waypoints, and control commands may become failure points that propagate across the P–C–A loop. Our analysis highlights structured output contracts, independent safety barriers, and deterministic fallback mechanisms as key design elements for the reliable integration of LLM capabilities into UAV platforms.