Domain-specific continual adaptation of LLMs risks catastrophic forgetting, creating a fundamental tension between acquiring new capabilities and preserving those learned during pretraining. PEFT mitigates this problem by restricting the number of trainable parameters, but existing methods lack a principled unit for de...
Ling-Xiang Wang, Hai-Nan Zhang, Liang Pang et al.· 0 citations
Experiments across three language models and two multimodal models show that PIECE maintains general capabilities and achieves state-of-the-art continual learning performance across diverse downstream tasks, highlighting a practical path to scalable, domain-adaptive foundation models without catastrophic forgetting.
This work proposes AURORA, a novel hallucination detection framework that shifts the focus from static representations to the weight-gradient dynamics of LLMs, and achieves strong hallucination detection performance across four model families and four benchmark datasets.
Z. Zhang, Hainan Zhang, Zhiming Zheng· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.