TL;DR
The conclusions demonstrate that the proposed method achieves lower forgetting, lower perplexity on previously learned domains and a better stability–plasticity trade-off than naive fine-tuning, LoRA and Elastic Weight Consolidation, while requiring comparable computational resources.
This work proposes FlowLess-R, a representation-space regularization method that constrains replay representations relative to stored references while allowing continued learning and introduces representation flux, a geometric measure of sample-level representation displacement across training.
Investigation of the influence of fully connected FC layer architecture on parameter regularization in the class incremental learning setting using a modified ResNet-18 trained on the CIFAR-10 dataset provides both a novel parameter regularization strategy and new insights into the interaction between network architect...
The results suggest that the combination of sparse representations, local learning, and persistent memory is a promising direction for continual learning, while motivating further investigation into the respective roles of learning rules, representations, and architectural design in mitigating catastrophic forgetting.
TASSO, a new paradigm that efficiently preserves the latent space geometry while ensuring network plasticity, is introduced with two complementary techniques: subspace learning and geometry-aware knowledge distillation.
This work observes that pooled token embeddings from a frozen LLM embedding layer already separate task distributions throughout the learning sequence, and concludes that a Gaussian mixture model fitted on these embeddings, without any gradient-based training, is sufficient for task-agnostic adapter selection at test t...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
