K. Filus

Sparse Autoencoders Can Induce Local Geometry for Mitigating Catastrophic Forgetting

This work introduces Sparse Autoencoder Geometric Anchoring (SAGA), which converts a Sparse Autoencoder (SAE) trained at each task boundary into local, low-rank constraints on representation drift, and establishes its local relation to latent code matching, define its spectral truncation error, and derive a conditional...