The Cell, Not the Call: A Preregisterable Audit Protocol for Repeated-Call Behavioral Experiments With Large Language Models
Abstract. When a hosted large language model answers the same prompt many times, the resulting completions can estimate that model's response distribution for that one prompt. They cannot, on their own, stand in for a sample of scenarios, participants, or models, and treating them as if they could undermines the claims...