The Reversed Turing Test: Measuring Superhuman AI Performance Beyond Imitation
TL;DR
It is argued that imitation-based evaluation is misaligned with general intelligence systems and the Reversed Turing Test (RTT) is introduced, which asks whether a machine can be identified precisely because its performance is measurably superhuman.
Abstract
For more than seven decades, Alan Turing’s imitation game has shaped how the AI community discusses machine intelligence, although it was never a psychometric instrument. AI systems now outperform humans in strategic games, scientific prediction, and pattern recognition, achievements better described as domain-specific superhuman performance than as general intelligence unless a novelty condition is enforced. This paper argues that imitation-based evaluation is misaligned with such systems and introduces the Reversed Turing Test (RTT), which asks whether a machine can be identified precisely because its performance is measurably superhuman. The test combines a standardized superhuman index (SI), expressed in standard-deviation units relative to a human reference sample, with a panel of judges whose identifications are verified against the performance evidence. The paper provides a formal definition of the RTT, a taxonomy of novelty types with verification methods, a worked example in protein structure prediction, and a pre-registered text-generation study (n = 500; composite SI ≈ 3.2; 90% identification accuracy grounded in stylometric cues), interpreted as superhuman stylometric performance rather than superhuman intelligence. The RTT is compared against established benchmarks, its philosophical limits are stated, and implications for trust, creativity, governance, and evaluation design are discussed.