Wasserstein-Regularized Low-Rank Adaptation (OTLoRA): Mitigating Representation Drift and Catastrophic Forgetting in Small Language Models
ABSTRACT: The decentralized deployment of Small Language Models (SLMs) in domain-specific downstream tasks requires continuous parameter adaptation while strictly preserving fundamental linguistic representations. Low-Rank Adaptation (LoRA) has emerged as the premier Parameter-Efficient Fine-Tuning (PEFT) paradigm due...