A Survey on the Linear Representation Hypothesis
TL;DR
It is argued that claims regarding linear representations become well-defined only through careful examination of the model, representation location, feature definition, and evaluation dataset, and a more rigorous formalization of the LRH is proposed that makes these dependencies explicit and allows the hypothesis to be evaluated as a falsifiable scientific claim.
Abstract
The term"linear representation hypothesis"(LRH) has appeared across diverse subfields of artificial intelligence, neuroscience, and cognitive science. But previous works have not consistently treated the LRH as a falsifiable scientific hypothesis; we analyze these inconsistencies and examine their implications for how prior theoretical and methodological results should be interpreted. Based on this analysis, we argue that claims regarding linear representations become well-defined only through careful examination of the model, representation location, feature definition, and evaluation dataset. We therefore propose a more rigorous formalization of the LRH that makes these dependencies explicit and allows the hypothesis to be evaluated as a falsifiable scientific claim. Finally, we identify some non-trivial open problems that warrant further attention from the research community.
