PR #5245 Set tp to 1 when MPU is None for bf16 optimizer

Set tp to 1 when MPU is None for bf16 optimizer #5245

samadejacobs wants to merge 6 commits into master from fix_mpu_ckpt

Set tp world size to 1 in ckpt load, if MPU is not provided

b75647c4

Set tp=1 where MPU is None for bf16 optimizer

283eda86

Set tp=1 where MPU is None for bf16 optimizer

782642d7

samadejacobs requested a review from

tjruwase 1 year ago

samadejacobs requested a review from

mrwyattii 1 year ago

Merge branch 'master' into fix_mpu_ckpt

ce4dafbf

Merge branch 'master' into fix_mpu_ckpt

37b78508

Formatting

f335cd59

Reviewers

tjruwase

mrwyattii

Assignees

No one assigned

Labels

None yet

Milestone

No milestone