PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer
PoLoRA is introduced, a Preconditioned Orthogonalized LoRA optimizer built from three ingredients: a product-aware spectral update direction, curvature preconditioning derived from controlling the per-sample loss change, and a magnitude rule that controls the sizes of both the factor and merged updates.