DeepSpeed
4388a605 - Update lr_schedules.py (#4563)

Commit
2 years ago
Update lr_schedules.py (#4563) add cosine annealing scheduler this scheduler is widely used in image classification task, and many llm (e.g. llama) use this also. --------- Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
Author
Parents
Loading