DeepSpeed
Separate ZeRO3 InflightParamRegistry for train and eval
#3884

Merged

Separate ZeRO3 InflightParamRegistry for train and eval #3884

HeyangQin merged 9 commits into master from HeyangQin/fix_pr_3462

extend the test and fix fp16 typo.

17228f43

guard reset params with z3 enabled check.

709bbdd4

Merge branch 'master' into fix-he-lora-z3-test

cbbba7c7

create standalone registries for training and eval respectively

badfb23a

HeyangQin requested a review from

jeffra 2 years ago

HeyangQin requested a review from

tjruwase 2 years ago

HeyangQin requested a review from

samyam 2 years ago

HeyangQin requested a review from

mrwyattii 2 years ago

make this pr standalone

1fcbd844

standalone v2

f270a2b4

HeyangQin requested a review from

RezaYazdaniAminabadi 2 years ago

HeyangQin requested a review from

awan-10 2 years ago

HeyangQin requested a review from

cmikeh2 2 years ago

HeyangQin requested a review from

arashb 2 years ago

Merge branch 'master' into HeyangQin/fix_pr_3462

419461b8

HeyangQin enabled auto-merge (squash) 2 years ago

awan-10 approved these changes on 2023-07-05

Merge branch 'master' into HeyangQin/fix_pr_3462

ec0d9eb2

Merge branch 'master' into HeyangQin/fix_pr_3462

cd2ba62f

HeyangQin merged 9377921a into master 2 years ago

mrwyattii deleted the HeyangQin/fix_pr_3462 branch 2 years ago

Reviewers

awan-10

jeffra

tjruwase

samyam

mrwyattii

RezaYazdaniAminabadi

cmikeh2

arashb

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed Separate ZeRO3 InflightParamRegistry for train and eval #3884 Merged

Separate ZeRO3 InflightParamRegistry for train and eval #3884

DeepSpeed
Separate ZeRO3 InflightParamRegistry for train and eval
#3884

Merged