Causal and Interpretable Structures in LLM Compositional Tasks
This work studies activations from ensembles of prompts that require inferring relationships between three tokens corresponding to a cyclic concept to correctly predict the next token and finds a consistent layerwise progression in how the joint dependence among the tokens is geometrically organized and causally used.