Online continual learning (OCL) requires a learner to acquire a sequence of tasks in a single pass over the data, and catastrophic forgetting of earlier tasks remains its central challenge. Despite the many methods proposed to mitigate forgetting, the theoretical understanding of forgetting in OCL is still limited: exi...
On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher's full-vocabulary distribution at every token is costly. Entropy-aware OPD (EOPD) adds forward supervision to reverse KL to help the student recover plausible tokens it...
Zi-Xiang Ni, Zhuo-Lun Hu, Ren-Jie Cao et al.· arXiv.org· 0 citations
This work forms a rank-aware information state combining local rank, scaled moment sensitivity, source-separated covariance, and a moment compatibility residual and reports missing or weak information instead of treating every fitted correction as identified.
Shu-Heng Cao, Zhen-Hao Zhang, Rui-Qi Chen et al.· 0 citations
This work introduces RCShift, which certifies two routes to sufficiency under a declared observation contract, and exact mode characterizes minimum-cost family-exact storage through LR-visible cycle directions through LR-visible cycle directions.
Shu-Heng Cao, Rui-Qi Chen, Zhen-Hao Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.