Normalized Low-Rank Adaptation
Normalized Low-Rank Adaptation (NoRA) is introduced, a simple yet effective method that normalizes the down-projection matrices during training, improving standard LoRA without requiring repeated normalization throughout training.
Jiale Kang, Zi-Yin Yue, Zheng Zhan et al.
· 0 citations