Identifying Crucial Attention Heads for Multilingual Language Models: Retrieval and Retrieval-Transition Heads
A cross-lingual needle-in-the-haystack task is designed, to identify RTHs that retrieve key information and further map it to target-language output and advances the understanding of multilingual LMs by isolating the attention heads responsible for mapping to target languages.
Shaswat Patel, Vishvesh Trivedi, Yue Han et al.
· 0 citations