peft

5e5a1b7d - FEAT Add LoRA-GA (#2926)

FEAT Add LoRA-GA (#2926) Adds "LoRA-GA: Low-Rank Adaptation with Gradient Approximation" Paper link: https://arxiv.org/abs/2407.05000 Initialize the LoRA weights to approximate full rank gradient updates, which promises better convergence.