Continuous Knowledge-Preserving Decomposition with Adaptive Layer Selection for Few-Shot Class-Incremental Learning
TL;DR
This work proposes CKPD-FSCIL, a unified framework for knowledge-preserving adaptation at both the weight and layer levels, and demonstrates state-of-the-art or competitive performance, together with favorable knowledge retention, scalability, and efficiency.
Abstract
Few-shot class-incremental learning (FSCIL) requires models to learn novel classes from limited examples while retaining previously acquired knowledge, leading to a fundamental stability--plasticity trade-off. Existing methods commonly freeze pretrained backbones, which restricts adaptation, or introduce prompts and adapters that may increase model size and inference cost across sessions. We study FSCIL from the perspective of directly reusing and adapting pretrained weights. To this end, we propose CKPD-FSCIL, a unified framework for knowledge-preserving adaptation at both the weight and layer levels. At the weight level, Continuous Knowledge-Preserving Decomposition uses past-class feature covariance to partition selected weights into preserved and learnable components, and recalibrates this partition across incremental sessions. At the layer level, Continuous Adaptive Layer Selection uses an Adapter Sensitivity Ratio to dynamically identify comparatively suitable layers for adaptation. After each session, the learned low-rank components are merged into the original weights, introducing no additional inference-time parameters or FLOPs. Experiments across multiple FSCIL benchmarks, backbone architectures, and evaluation settings demonstrate state-of-the-art or competitive performance, together with favorable knowledge retention, scalability, and efficiency. The code is available at https://github.com/xiaojieli0903/CKPD-FSCIL.