P-Hacking Should Not Make Us Abandon the Principle of Total Evidence

Abstract

Practices such as “p-hacking” or “fishing” are widely regarded as epistemically pernicious, raising a puzzle. If two researchers generate identical data sets, why should it matter that one “fished” for the result whereas the other predicted it? Some respond to this puzzle by abandoning the Principle of Total Evidence (PTE), proposing that we exclude “tainted” data or weaken our descriptions of evidence. I argue that such departures are unnecessary and counterproductive. The most common forms of p-hacking are pernicious precisely because they violate PTE. For the remaining cases, I demonstrate that suppressing evidence leads to absurd results.