Active Inference with People: a general approach to real-time adaptive experiments

Abstract

Adaptive experiments optimize their design throughout data collection, which can bring substantial benefits compared to conventional experimental settings. Potential applications include, among others, computerized adaptive testing (when selecting informative tasks in ability measurements), adaptive treatment assignment (when searching for experimental conditions maximizing certain outcomes), and active learning (when choosing optimal training data for machine learning algorithms). However, implementing these techniques in real time poses substantial computational and technical challenges. In this paper, we introduce a practical and unified approach to real-time adaptive experiments that can encompass these scenarios across textual, visual, and audio tasks. Our strategy combines active inference, a Bayesian framework inspired by cognitive neuroscience, with Pyro, a probabilistic programming library, and PsyNet, a modular Python package for large-scale online behavioral experiments. Active inference provides a task-agnostic optimization objective and efficient inference strategies; probabilistic programming makes the computations practical, reducing implementation costs; and PsyNet makes the resulting procedure deployable with humans in real time across diverse behavioral paradigms. We illustrate this approach through two concrete examples: (1) an adaptive testing experiment estimating participants' ability by selecting optimal challenges, reducing the number of trials required by 30--40\%; and (2) an adaptive treatment assignment strategy that identifies the optimal treatment up to three times as accurately as a fixed design. We provide instructions to facilitate adoption of the workflow.