A Comprehensive Evaluation of Generative Models for Privacy-Preserving Synthetic Student Data

Abstract

Privacy regulations and institutional policies limit the sharing of educational data, constraining reproducibility in learning analytics. Prior evaluations of synthetic data on benchmarks such as OULAD have examined statistical or adversarial synthesizers in isolation, rarely jointly assessing fidelity, utility, privacy, and explainability. We compare three synthesis paradigms, statistical (Gaussian Copula), adversarial (CTGAN), and diffusion-based (TabDDPM), on two benchmarks (OULAD: 32,593 records; ASSISTments: 8,519) across five evaluation axes: distributional fidelity (SDMetrics), downstream utility (Train on Synthetic, Test on Real), discriminative realism (classifier two-sample test), membership-inference privacy, and feature-importance preservation (SHAP). The pipeline is repeated over five random seeds with bootstrap confidence intervals and Bonferroni-corrected permutation tests (<inline-formula> <tex-math notation="LaTeX">$\alpha \prime \approx ~0.0028$ </tex-math></inline-formula>). Four findings emerge: First, TabDDPM delivers the strongest classification utility: on OULAD, a Random Forest achieves TSTR AUC <inline-formula> <tex-math notation="LaTeX">$= 0.962~\pm ~0.001$ </tex-math></inline-formula>, within 0.5 percentage points of the real-data baseline. Second, all synthesizers exhibit near-chance membership-inference risk under the evaluated kNN-based threat model (worst-case effective AUC <inline-formula> <tex-math notation="LaTeX">$\le 0.527$ </tex-math></inline-formula>). Third, distributional fidelity does not predict task utility; CTGAN scores highest on SDMetrics yet does not yield the smallest utility gap. Fourth, TabDDPM best preserves real-data feature-importance rankings on OULAD (Spearman <inline-formula> <tex-math notation="LaTeX">$\rho =0.846$ </tex-math></inline-formula>, p < 0.001). ASSISTments SHAP correlations (<inline-formula> <tex-math notation="LaTeX">$\rho ~ \ge 0.950$ </tex-math></inline-formula>) reflect a low-dimensionality ceiling effect rather than meaningful synthesis quality differences. These results provide task-driven guidance for selecting a synthesizer in learning analytics. Scope is limited to static tabular benchmarks; temporal, sequential, multimodal, and fairness-aware synthesis remain outside the present scope. Runtime results are based on CPU execution, so neural synthesizers may run faster under GPU acceleration.