Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures
Overall, query-dependent cross-attention is the most reliable mechanism, whereas branch self-attention is most useful for large, spatially complex functional inputs, whereas branch self-attention is most useful for large, spatially complex functional inputs.
Amar Alem Koric, Qi-Bang Liu, S. Koric
· 0 citations