An equality condition for the Dobrushin bound on attention rollout and how often it holds in trained transformers
This work characterises exactly when the layerwise bound is tight: equality holds if and only if some token pair attaining $\kappa(A)$ is mutually self-dominant - each of the two attends to itself at least as strongly as the other attends to it.