Benzi Busigin

Inside the LLM Word Factory

This work ventures deep into detokenization using activation patching in controlled paired experiments that isolate the contribution of different model components, localizing English detokenization in Llama2-7B to a two-stage process at Layer 1.