Jul 2026· Italian National Conference on Sensors· Vol 26, pp. 4232· 0 citations· 45 references
Medicine
TL;DR
HRL-Det, a hierarchical reinforcement learning framework that addresses object detection in unmanned aerial vehicle (UAV) imagery through two tightly coupled innovations, and constructs convergence-promoting dense rewards informed by Lyapunov stability analysis while maintaining exploration diversity through maximum entropy optimization.
Abstract
Object detection in unmanned aerial vehicle (UAV) imagery suffers from severe scale variation, dense object packing, and prohibitive computational cost when conventional detectors exhaustively evaluate high-resolution frames. Reinforcement learning (RL)-based sequential detectors offer a promising alternative by formulating localization as an active search process, yet existing methods are limited by discrete-time state transitions, sparse reward signals, and premature policy collapse. In this paper, we propose HRL-Det, a hierarchical reinforcement learning framework that addresses these challenges through two tightly coupled innovations. First, a Neural ODE-driven Continuous-Time Bellman State Evolution module models the agent’s state dynamics as a stochastic differential equation governed by the Hamilton–Jacobi–Bellman equation, enabling fine-grained temporal reasoning with memory-efficient adjoint-based backpropagation. Second, a Lyapunov-Guided Entropy-Regularized Reward Shaping mechanism constructs convergence-promoting dense rewards informed by Lyapunov stability analysis while maintaining exploration diversity through maximum entropy optimization. Extensive experiments on VisDrone2019, DroneVehicle, and MS COCO 2017 show that HRL-Det achieves mAP@0.5 of 0.412, 0.812, and 0.735, respectively, outperforming existing RL-based detectors and achieving competitive accuracy relative to representative non-RL detectors under the same COCO metric, while requiring only 17.3 M parameters and an average of 6.3 search steps per object.
Safe quadrotor navigation in cluttered and dynamic environments depends not only on instantaneous geometric perception, but more critically on anticipating collision risks induced by relative motion. Conventional modular pipelines frequently suffer from perception latency, while end-to-end learning methods relying on implicit scalar rewards often struggle to extract reliable spatio-temporal features without physics-grounded supervision. To address this, we propose an anticipatory risk-guided reinforcement learning framework. Leveraging privileged simulator states, we construct a directionally aligned future collision risk map based on the Closest Point of Approach (CPA). Through an asymmetric actor-critic architecture, the network is trained to self-predict this structured risk, which explicitly guides the visual policy during deployment. A lightweight spatio-temporal encoder extracts motion cues directly from onboard depth sequences, bypassing explicit object tracking or optical flow estimation. Extensive simulated and real-world experiments demonstrate that our method effectively improves safety margins and flight efficiency in dense dynamic clutters compared to existing baselines. Furthermore, the learned policy achieves robust zero-shot Sim-to-Real transfer on a physical quadrotor, relying purely on abstracted spatio-temporal depth sequences and its self-predicted risk priors, validating the effectiveness of our approach and its robust generalization from simulation to reality.
Yuchao Mei, Guohao Zhang, Luxia Ai et al.· arXiv.org· 0 citations
Visual localization is vital for autonomous systems but remains challenging under dynamic conditions. Transformers offer strong temporal modeling at quadratic cost, while CNNs are efficient yet limited in long-range dependencies. Existing methods also lack robustness to illumination, weather, and seasonal changes, constraining real-world applicability. To address this, this paper proposes AdapseqNet, a dual-branch architecture that integrates stabilized state-space modeling with differential temporal enhancement. First, a stabilized state-space formulation featuring Lyapunov-constrained parameterization and adaptive discretization is proposed, ensuring asymptotic stability and linear computational complexity for reliable processing of extended sequences. Second, a selective Mamba architecture is developed to combine temporal-state modeling with content-aware gating, enabling adaptive feature selection that emphasizes discriminative cues while suppressing redundancy. Third, a differential enhancement module is designed to extract motion-invariant representations through symmetric temporal differencing and LSTM-based refinement, enhancing resilience to appearance variations caused by lighting, weather, and seasonal changes. Beyond architectural design, multi-scale feature fusion and output distribution control are incorporated to optimize representation quality and ensure consistency for similarity-based retrieval. Extensive experiments on multiple benchmarks demonstrate that AdapseqNet achieves a better localization accuracy across diverse and challenging conditions. Note to Practitioners—Visual localization is crucial for autonomous robots but often fails under varying lighting, weather, or seasonal conditions. We propose a dual-path approach: one path captures long-term patterns using control-inspired stable modeling, while the other extracts motion cues that remain consistent despite appearance changes. This combination enables accurate place recognition even in extreme environments. Our system operates efficiently on standard hardware and was tested on an indoor robot, achieving centimeter-level accuracy. This approach can enhance existing navigation systems without requiring additional sensors. Future work will focus on real-time optimization for outdoor deployment.
Zhenyu Li, Tian-Yi Shang· IEEE Transactions on Automat...· 0 citations
Reinforcement learning is promising for autonomous urban driving, but long-horizon goal-directed navigation asks a policy to acquire several competing behaviors at once--reaching a distant goal, tracking a route, avoiding obstacles, obeying signals--and a fixed objective gives no order in which to learn them. This paper presents CORAL, which advances two schedules together: a five-stage curriculum that progressively lengthens routes and tightens behavioral constraints, and a stage-aware reward whose component weights shift emphasis from mission progress toward route following, safety, smoothness, and rule compliance as the task hardens. The policy is a multi-stream actor-critic network trained with Proximal Policy Optimization (PPO) in CARLA on a compact 99-dimensional state pairing a polar LiDAR histogram with vehicle telemetry, ego-frame route geometry, and traffic-rule indicators--no point-cloud encoder, no bird's-eye-view rasterization. Against two PPO baselines under an identical protocol, CORAL reaches the goal in all twenty evaluation episodes on the longest routes under the full set of behavioral constraints, where the baselines reach 5% and 10%; a factorial ablation shows that neither schedule alone matches their combination: removing either lowers both success and route completion, and disabling both drops success to 55%. Trained in one town, the policy transfers zero-shot to seven unseen towns, succeeding in 68-98% of episodes on routes of the same 100-150 m length, with mean lateral deviation below 0.35 m.
DAR-Track is proposed, a novel framework that harmonizes dynamic computation with generative modeling and outperforms state-of-the-art methods, including MixFormer and SGLATrack, while maintaining superior inference speeds suitable for real-time aerial robotics.
Wenqin Dong· International Conference on...· 0 citations
Autonomous navigation in cluttered environments is hampered by partial observability and dynamic constraints. This paper presents PILOT, a constraint-aware privileged imitation learning framework for vision-based end-to-end UAV motion planning under partial observability. The framework distills planning strategies from a computationally intensive optimal control expert into a student policy regularized toward safety and dynamic requirements via a dual-objective loss function. To mitigate partial observability, a spatiotemporal perception fusion module using a Temporal Convolutional Network (TCN) is developed to integrate historical depth images and odometry. This module infers task-relevant latent context from historical observations, enhancing spatial awareness beyond the instantaneous FOV without maintaining persistent map memory. A trajectory parameterization layer mapping network outputs to a structured trajectory, while enabling explicit continuity, dynamic-consistency, and obstacle soft penalties during training, encouraging constraint satisfaction for unseen observations without formal guarantees. Simulations on quadrotor and fixed-wing aircraft demonstrate that PILOT achieves performance comparable to the privileged expert while reducing computational overhead by over 80\%. Successful indoor and outdoor zero-shot deployment confirms the practical feasibility and cross-domain generalization of the planner.
Qing-Rui Zhang, Feng Xue, Xiang Zhou et al.· 0 citations
A systematic classification of recent breakthroughs in real-time perception, specifically evaluating stereo image processing and Simultaneous Localization and Mapping through the lens of computational economy, and analyzes the efficacy of specialized optimization frameworks designed to maximize inference speed on embedded CPU/GPU architectures.
R. Abri, Sara Abri, S. Cetin· International Conference on...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
