HALAS: On-Demand and AoI-Guided Explainability for Real-Time XR Analytics.
Abstract
Understanding user behavior in extended reality (XR) is often performed through post-hoc pipelines that log complete sessions and invoke large language models (LLMs) offline. Such pipelines incur substantial latency, bandwidth overhead, and unnecessary LLM inference costs, limiting continuous in session analytics. We present HALAS, a hierarchical, Analysis of-Interest (AoI)-guided analytics system that enables on-demand XR behavior analysis through a cloud-edge-device workflow. HALAS consists of (i) an on-device AoI-aware controller that estimates posterior AoI relevance from observed behavior evidence and retains behaviors that reach a calibrated threshold; (ii) edge-hosted lightweight models that generate low-latency context descriptions and behavioral insights for routine XR behaviors; (iii) a token-confidence gate that selectively routes uncertain cases to cloud LLMs through cascading inference; and (iv) a referent-delta transmission scheme that reuses transmitted object representations and sends incremental updates. Across controlled XR settings, HALAS maintains a mean first-insight B2I of 71.9 88.0 ms and 1.37-1.45 MB of uplink traffic. Its mean CoAScore remains within 0.19-0.30 points of the Na¨ ıve Online LLM and Explainable XR baselines, while avoiding their 6.3 s and 33.9 s de lays. HALAS achieves comparable insight quality to large-model processing while providing millisecond-scale responsiveness and low transmission overhead across the XR environments. The code is available at: https://github.com/liboyang696-debug/HALAS.