DeepSpeed
abe4fc6b - encoded ds config into command line argument when launching child processes in autotuning (#2524)

Comment changes are shownComment changes are hidden
Commit
2 years ago
encoded ds config into command line argument when launching child processes in autotuning (#2524) * rollback ds config changes * fix format * Fix error when output_file is a relative path without a prefix (#2397) Co-authored-by: Benjamin Steenhoek <benjaminjsteenhoek@gmail.com> * fix restuls and exprs path to use absolute path * use base64 encoded ds config as cmd arg * fix format * remove assert * write out optimial config after tuning * fix format * no need to update ds config path when encoding ds config * udpate * do not use abs path for result and expr dir * fix conflicts * fix run mode * fix format * fix format Co-authored-by: Benjamin Steenhoek <benjaminjsteenhoek@gmail.com> Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Author
Parents
  • deepspeed
    • autotuning
      • File
        autotuner.py
      • File
        config.py
      • File
        scheduler.py
    • runtime
      • File
        config.py
      • File
        engine.py