Using OCR Heads to Verbalize Image Semantics
Across four models, attention heads causally necessary for OCR are identified, and it is discovered that these are in fact general-purpose heads that output interpretable semantic features across all image tokens.