DeepSpeed
Separate ZeRO3 InflightParamRegistry for train and eval
#3884

Merged

Commits

extend the test and fix fp16 typo.

awan-10 committed 2 years ago
guard reset params with z3 enabled check.

awan-10 committed 2 years ago
Merge branch 'master' into fix-he-lora-z3-test

awan-10 committed 2 years ago
create standalone registries for training and eval respectively

HeyangQin committed 2 years ago
make this pr standalone

HeyangQin committed 2 years ago
standalone v2

HeyangQin committed 2 years ago
Merge branch 'master' into HeyangQin/fix_pr_3462

HeyangQin committed 2 years ago
Merge branch 'master' into HeyangQin/fix_pr_3462

HeyangQin committed 2 years ago
Merge branch 'master' into HeyangQin/fix_pr_3462

mrwyattii committed 2 years ago