Multimodal Physiological and Behavioral Correlates of Situation Awareness in Virtual Reality Flight Navigation

Abstract

Situation awareness (SA) changes as task conditions evolve, yet commonly used measures capture it at discrete times. This has motivated interest in physiological signals as continuous indicators of SA. Although prior work suggests these signals relate to SA, studies have often examined isolated modalities in limited task environments, while recent work indicates that physiological relationships may differ across SA outcomes, including accuracy and response time (RT). This study examined multimodal physiological and behavioral correlates of SA outcomes in 115 U.S. Naval Aviation trainees completing the Assessing Spatial Abilities in Naval Aviation (ASANA) immersive virtual reality flight navigation task. The results showed distinct patterns. Higher accuracy was associated with efficient navigation, higher heart rate, more structured gaze transitions, and faster head scanning. Shorter RT was associated with efficient navigation and lower heart rate variability. These findings highlight multimodal physiological and behavioral measures in immersive environments for characterizing SA.