Curriculum learning support (#132)
* CL initial commit
* CL+PP support
* update
* Apply suggestions from code review
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* apply code review comments
* make it easier to read large numbers
* add a cl test
* apply review comments
* Update examples/curriculum_learning/README.md
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* update
* fix
* new requirement
* Update megatron/learning_rates.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Update megatron/learning_rates.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* fix samples and tokens - thank you Conglong
* fix truncation
* switch to deepspeed@master
* extend the doc
* Trigger CI
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas@stason.org>
