Abdul Kadir

Finding the Heads and the Neurons Responsible for Network Information Retrieval in Language Models

We ask whether specific attention heads, and more finely specific neurons inside those heads, are responsible for recognizing that a language model's context contains network infrastructure information (a hostname paired with its IP address), and whether that responsibility can be validated causally rather than by corr...