DeepSpeed
f994fb2c - Update LR scheduler configuration (#5846)

Commit
1 year ago
Update LR scheduler configuration (#5846) This PR is based on https://github.com/microsoft/DeepSpeed/issues/5726. The current lr scheduler initialization always prioritize config over manual defined scheduler in the code. However, the optimizer initialization implementation prioritize manual defined optimizer over config. This PR aims to make initialization behavior for both optimizer and lr scheduler consistent where if lr scheduler is defined in the code, then it will overwrite config. --------- Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Parents
Loading