S-FUEL: A Semantic and Latency-Aware Framework for Target-Oriented UAV Exploration Using Onboard Foundation Models

Abstract

Efficient exploration and target identification in unstructured environments are critical challenges in UAV automation. While foundation models like Segment Anything Model 3 (SAM 3) offer powerful open-vocabulary perception, their high computational cost and inference latency hinder real-time deployment on onboard hardware. Traditional geometric methods, conversely, ensure safety but lack semantic awareness. This paper presents the Semantic-Foundation UAV Exploration with Latency-awareness (S-FUEL) framework, which bridges this gap via an asynchronous semantic-geometric fusion strategy. We formulate the exploration task as a dual-layer optimization problem. First, to handle the low-frequency semantic updates, we introduce a Latency-Aware Semantic Utility Function, which asynchronously updates global goals based on SAM 3’s open-vocabulary detections without blocking the high-frequency local planner. Second, to ensure safety against dynamic obstacles despite sparse detections, we propose a Predictive Semantic Repulsion Field. This module leverages SAM 3’s video tracking memory to forecast obstacle trajectories and incorporate spatiotemporal priors into the local map, enabling the UAV to proactively avoid dynamic threats between semantic frames. Experimental results demonstrate that S-FUEL reduces “time-to-target” by about 42% compared with the FUEL geometric baseline and remains 10.6% faster than the strongest adapted semantic baseline. At the evaluated 50 Hz/2 Hz planning–semantic rate pair, the bounded predictor yields a 2% nominal collision rate; the rate rises to at most 4% under the tested weight perturbations, so the method reduces rather than eliminates dynamic-obstacle risk. Note to Practitioners—This study addresses a pressing practical challenge in autonomous UAV operations: enabling real-time, target-oriented exploration using computationally heavy foundation models on resource-constrained onboard hardware. In real-world search and rescue or industrial inspection missions, a fundamental mismatch exists between the high-speed requirements of flight control and the significant latency of advanced vision models like SAM 3. This latency often creates “blind intervals” where the UAV is unaware of obstacle movements between semantic updates. To overcome this, we propose the S-FUEL framework, which decouples semantic reasoning from motion planning through an asynchronous dual-thread architecture. Our method allows the UAV to utilize high-level semantic guidance for efficient searching without compromising local safety or control stability. The proposed strategy is particularly applicable to autonomous systems operating in dynamic, unstructured environments where rapid target identification and high-frequency collision avoidance are both essential.