Stop collecting all model parameters to save models when using DeepSpeed and LoRA (#41416)
no need to gather all parameters to save model when using deepspeed and LoRA
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Ferdinand Mom <47445085+3outeille@users.noreply.github.com>