This work examines two representative architectures, MAtt and GBWAtt, and finds that their learned attention weights remain close to uniform after training, and introduces SPD-MetaFormer, an attention-free architecture built on uniformly weighted Fr\'echet aggregation under log-Euclidean geometry.
It is proved that the constrained decoupled objective is equivalent at optimality to a specific joint penalty on the inner and outer weights, for any positively homogeneous activation, and thus admits a clean projected and proximal formulation.
Charles Kulick, Armenak Petrosyan, Tang Sui· 0 citations
The Projected Koopman Operator Approximation framework for constructing Filtered EDMD operators is introduced, and under independent noiseless sampling and exact-rank identifiability, the resulting empirical operators converge almost surely to their population counterparts.
Si-Jin Chen, Igor Mezić, Tang Sui· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.