Inference and learning in sparse autoencoders as natural gradient flow
Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. These challenges involve both inferring which features explain an input and learning the dictionary that represents them. Here, we unify infer...
Hadi Vafaii, Tejas Rao, David Chanin et al.
· 0 citations