// Normalized Low-Rank Adaptation (NoRA) // They propose a one-line change to LoRA that costs nothing and improves convergence, stability and forgetting. LoRA initializes the up-projection to zero, which means early optimization is governed almost entirely by the down-projection. This observation tells you where to regularize. NoRA normalizes the down-projection matrices during training. The authors also show the same normalization applied once at initialization improves standard LoRA…