Caio F. Deberaldini Netto

Attention Graphons: A Graph Limit Perspective on Graph Transformers

Graph Transformers produce, for each attention head, a dense $n\times n$ matrix of learned pairwise interactions. We ask a fundamental question: do these attention-induced graphs converge to a stable limit object as $n$ grows, or does the learned interaction pattern remain unstructured and size-dependent? We answer thi...