Manipulative Gesture Patterns in VR Hand Tracking under Gravity and Non-Gravity Conditions: An Exploratory Study with Children and Young Adults

Abstract

Virtual reality (VR) hand-tracking systems offer novel affordances for gesture-based object interaction, including conditions impossible in the physical world such as the absence of gravity. However, how these affordances shape manipulative gesture production across developmental groups remains underexplored. This exploratory study examines goal-directed manipulative gestures elicited during a spatial construction task (tangram puzzle) in VR hand-tracking under two conditions: with and without gravity. Seven children (aged 8–14) and seven young adults (aged 19–23) completed the task using an Oculus Quest 2 with built-in hand tracking. We applied and extended a manipulative gesture taxonomy comprising eight goal-directed gesture types, adding a Fine-tune category that emerged from observation. Across 3,170 coded goal-directed gestures, we found that adults produced more align and fine-tune gestures across both conditions, while children produced more move gestures. Both groups produced more rotate gestures in the non-gravity condition, with this pattern more pronounced in children. Hand kinematics analysis revealed that children’s two-finger pinched-O rotations clustered overwhelmingly in the non-gravity condition. We discuss design implications for VR hand-tracking applications targeting children, and methodological considerations for applying gesture taxonomies across interaction modalities.