An Investigation of Model Coherence: Narrow Finetunes Contradict Themselves Under Resampling
A large body of research measures model coherence based on output variance without adequately considering competing causes. We identify two such causes, ambiguity and indifference, and we introduce a set of 175 questions where contradicting answers cannot easily be explained by either. We then measure incoherence in te...