DeepSpeed
Update to new torch grad hook API: BF16Optimizer and Stage2
#7189
Merged

Commits
  • Avoid graph break by removing redundant requires_grad attr change
    deepcharm committed 1 year ago
  • Small fix
    deepcharm committed 1 year ago
  • Revert "Small fix"
    deepcharm committed 1 year ago
  • fix leak of z3 buffer
    deepcharm committed 1 year ago
  • hf tp+zero training doc. (#7151)
    deepcharm committed 1 year ago
  • Add destroy to tests to free memory (#7160)
    deepcharm committed 1 year ago
  • [NFC] Typo fix in SP layer. (#7152)
    deepcharm committed 1 year ago
  • Link AutoTP blog in the front page (#7167)
    deepcharm committed 1 year ago
  • fix `seq_parallel_communication_data_type` constant. (#7175)
    deepcharm committed 1 year ago
  • Fix typos in GDS blog (#7177)
    deepcharm committed 1 year ago
  • Variable batch size and LR scheduler (#7104)
    deepcharm committed 1 year ago
  • Update version.txt after 0.16.5 release (#7180)
    deepcharm committed 1 year ago
  • Cross layer overlapping for domino (#7178)
    deepcharm committed 1 year ago
  • async tp allreduce (#7115)
    deepcharm committed 1 year ago
  • Fix issue #5242 grad_norm and loss is nan (#7171)
    deepcharm committed 1 year ago
  • Update to new torch grad hook API: BF16Optimizer and Stage2
    deepcharm committed 1 year ago
  • Merge branch 'master' into use-new-grad-acc-api
    deepcharm committed 1 year ago
Loading