Bi-Compatible Task-Agnostic Feature Augmentation for Expansion-Based Class-Incremental Learning

Miles: Metric Learning With Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning

Miles decouples the learnable modules with the pre-trained model, exploiting prior information from intermediate features of the backbone network to enable more flexible parameter expansion, and orchestrating an efficient expansion of the parameter space through guided optimization.

MOBO: A Merging-Oriented Bi-Level Optimization Framework for Class Incremental Learning

Preserving Fairness in Knowledge Transfer for Exemplar-Free Class-Incremental Learning via Semantic Propagation

As a paradigm in continual learning, class incremental learning (CIL) aims to assimilate tasks with mutually exclusive label spaces in sequence while preserving previously established knowledge. Mitigating forgetting in CIL fundamentally relies on transferring knowledge across tasks. A straightforward exemplar-based approach promotes balanced knowledge transfer by replaying an equal number of samples from each old class. However, in the more challenging exemplar-free setting, this balance cannot be ensured because distillation-based cross-task knowledge transfer tends to focus more heavily on the knowledge acquired from the most recent tasks. To address the unfairness in knowledge transfer, we analyze the mechanisms underlying dark knowledge and introduce a Semantic Enhanced Knowledge Transfer (SEKT) method for exemplar-free CIL. Specifically, SEKT adopts a bi-flow framework. The first flow is the Semantic Guidance Flow (SGF), which is inspired by knowledge distillation and produces latent semantic distributions from the outputs of the old model to guide the new model toward generating similar distributions. The second flow is the Semantic Propagation Flow (SPF), which propagates latent early knowledge to the current task in order to mitigate the unfairness in knowledge transfer. SPF constructs a cross-task semantic similarity graph using aligned intermediate representations to enable semantic propagation. It employs an expert network to learn the pattern of semantic propagation, enabling real-time and stable semantic recovery during training. In contrast to the SGF that is more effective for transferring recent knowledge, the SPF learns complementary early knowledge through a semantic complementarity constraint. Moreover, the SPF is robust to noisy semantics, as the learned semantic distribution is regularized with an $\ell _{2,1}$ norm. Extensive experiments conducted on six datasets demonstrate the superiority of the proposed SEKT over existing exemplar-free CIL approaches.

One Adapter, Many Tasks: Task-Conditioned Feature Transformations for Continual Learning

FACET proposes an efficient replay-free task-conditioned feature consistency loss, aiming to mitigate catastrophic forgetting of the learned mixture distribution in the adapter's feature space, and demonstrates robust scalability.

Meta-Exploiting Complementary Semantic Consistency for Cross-Domain Few-Shot Learning Promotion

Meta-learning has emerged as an effective solver for cross-domain few-shot learning (CD-FSL) tasks. Despite achieving obvious progress recently, the typical episodic learning paradigm often causes the feature embedding model collapsing into the simplicity bias pitfall, viz., the model tends to prioritize some shortcut patterns (e.g., color, style, background) that are only sufficient to distinguish categories in source domain, while fail to generalize across domains. To mitigate this problem, we present a novel meta-learning framework which emphasizes meta-exploiting inductive bias to alleviate simplicity bias for CD-FSL promotion, and mainly contributes in the following four aspects. 1) We establish a novel inductive bias for CD-FSL, termed complementary semantic consistency (CSC). The rationale behind lies in that forcing the semantic consistency between two complementary feature learning schemes is beneficial to distill cross-domain transferable features. 2) We establish a solid theoretical foundation, supported by rigorous mathematical proofs and key lemmas, which demonstrates that CSC establishes a tighter generalization bound and facilitates the learning of domain-invariant features. 3) Inspired by CSC, we propose a general meta-learning framework, which implements complementary feature embedding models using parallel networks with the same architecture but different input forms, and introduce proper knowledge distillation losses to encourage the semantic consistency between different branches during meta-training. This framework can be seamlessly integrated with any complementary feature learning schemes. 4) To clarify this point, we instantiate two effective meta-learners based on the proposed framework. The former establishes a two-branch network that simultaneously classifies both the query image and its random local crops. The latter decomposes the query image into high-frequency and low-frequency components, which are then integrated into a parallel feature embedding network for category prediction, analogous to the original query image. Subsequently, a KL divergence based knowledge distillation loss is separately leveraged to force the prediction consistency between the complementary branches (e.g., local-global, spatial-frequency) during meta-training. By doing these, both learners are able to distill cross-domain transferable features with better generalization performance. Empirical results on diverse benchmarks consistently affirm the proposed framework’s advantages, while additional analysis provides compelling support for our key claims.

Complementary Asymmetric Representation Learning for Exemplar-Free Class-Incremental Learning

The proposed CARL framework employs two parallel ARL branches and aggregates their Mahalanobis distances during inference to improve prediction performance and is compared with recent methods using three widely recognized datasets.