Channel Gains to Captions: Task-Unified Multi-Level RF Sensing With Vision–Language Models
TL;DR
A generative formulation for RF sensing, where millimeter-wave/terahertz channel gains are mapped to captions describing multi-level environmental semantics, and this framework achieves superior performance with a broader semantic scope, and enables task-unified sensing beyond predefined tasks.
Abstract
This letter investigates a task-unified multi-level radio-frequency (RF) sensing framework driven by vision–language models (VLMs). Existing RF sensing methods rely on task-specific designs and provide only partial environmental information, limiting their ability to handle emerging 6G applications. To address this, we propose a generative formulation for RF sensing, where millimeter-wave (mmWave)/terahertz (THz) channel gains are mapped to captions describing multi-level environmental semantics. The framework solves this problem through a complementary design for RF-environment semantic bridging, where a VLM is fine-tuned to leverage its multimodal representations and prompt-conditioned semantic generation capabilities. Hence, different sensing tasks are specified through textual prompts, enabling the framework to handle diverse tasks in a unified manner. For fine-tuning, we introduce prompt-routed low-rank adaptation (LoRA) experts to achieve level-aware adaptation. Simulation results show that, compared with baselines, our framework achieves superior performance with a broader semantic scope, and enables task-unified sensing beyond predefined tasks. Under an unseen sensing requirement, it achieves an average F1-score improvement of 0.22 over the most competitive variant.