Beyond Metrics: A Human Validation Study of Procedural Personas in a Continuous Game Arena

Abstract

This dataset accompanies the paper “Beyond Metrics: A Human Validation Study of Procedural Personas in a Continuous Game Arena”. It contains the gameplay videos used as stimuli in the user study, the training plots for the five reinforcement‑learning personas, the raw survey data, and the complete statistical analysis output. All materials are provided to facilitate reproducibility of the reported results. File overview baseline.mp4, runner.mp4, survivalist.mp4, monster-killer.mp4, treasure-collector.mp4: The five 30‑second gameplay clips shown to participants in the within‑subjects forced‑choice identification and believability study. Each video features one of the five procedural persona agents (Baseline, Runner, Survivalist, Monster Killer, Treasure Collector) acting in the evaluation arena described in the paper. The clips are exactly the stimuli presented under Greek‑letter labels (Alpha–Omega). training-plots.7z: Compressed archive containing per‑persona TensorBoard training plots (cumulative reward, episode length, policy entropy, extrinsic reward, extrinsic value estimate, policy loss, and value loss) for all five personas over 1 million training steps. raw_data.xlsx: The fully anonymized raw survey responses (exported from Google Forms), including demographics, forced‑choice identification answers, believability ratings, and consistency ratings for the 34 participants retained after attention‑check exclusion. 100-episodes-evaluation-statistics.7z: Statistical data and evaluation related to 100 episodes for each five procedural persona agents. human_validation_analysis_code_and_data.7z: Data wrangling and statistical application to support data generation related to the human validation. analysis_report.txt: Human‑readable output of the Python analysis pipeline that reproduces all descriptive statistics, confusion matrices, and exploratory correlations reported in the paper. wrangle.py, analysis.py, report.py, schema.py: Code support to generate statistics and analysis report from the raw_data.csv. raw_data.csv: CSV version of raw_data.xlsx. ratings.csv, participants.csv: Tidy data generated automatically from raw_data.csv. README.md: Instructions to run the code. analysis_results.json: Machine-readable json of analysis_report.txt. uv.lock, pyproject.toml, mise.toml: Setup files. All personal information has been removed from the survey data; the videos and plots contain no identifying metadata. The training code, Unity project, and configuration files are available in the companion GitHub repository: https://github.com/Re-G3X/ml-agents/releases/tag/paper-aiide-2026