Pose-to-Command: A Personalized Embodied Control Layer for Ordinary Games

Abstract

Ordinary digital games are typically controlled through low-movement inputs such as touch, keyboards, or gamepads, limiting embodied and adaptive interaction. We present a personalized embodied control layer that allows users to remap ordinary games through self-defined body poses and short motions captured with commodity mobile devices. The system supports user-specific pose recording, lightweight temporal modeling, partial landmark tolerance, and cross-platform command realization for Android touch gestures and desktop keyboard input. We evaluate the approach through a technical benchmark and a within-subject user study. The benchmark shows that the pipeline is feasible for interactive use on commodity hardware, achieving strong recognition performance and low end-to-end latency. In the user study, we did not detect significant differences between self-defined and imposed mappings on the primary objective or subjective measures. In contrast, performance and ratings improved consistently across rounds, indicating that early use was shaped strongly by familiarization. Interviews nevertheless suggested that many participants experienced self-defined mappings as more natural and intuitive. Together, these findings position personalized embodied control as a practical cross-platform interaction layer for ordinary games, identify familiarization as the clearest empirical factor shaping first-session use, and generate the hypothesis that the benefits of personalization may emerge after users move beyond the earliest familiarization phase.