DeepSpeed
Fix potential memory issues when use deepspeed Z3
#6726

Merged

Commits

Set "ds_grads_remaining" to 0 when module doesn't have this variable

wenbinc-Bin committed 1 year ago
Set "__n_available_params" to 0 in release_and_reset_all()

wenbinc-Bin committed 1 year ago
Add unit stage3 test for running model twice in one step

wenbinc-Bin committed 1 year ago
Merge branch 'master' into fix_z3

tjruwase committed 1 year ago
Merge branch 'master' into fix_z3

loadams committed 1 year ago
Merge branch 'master' into fix_z3

loadams committed 1 year ago
Merge branch 'master' into fix_z3

loadams committed 1 year ago
Fix dtype error

wenbinc-Bin committed 1 year ago
Merge branch 'master' into fix_z3

hwchen2017 committed 1 year ago