PR #5145 [Dist] Refactor ZeRO-1

[Dist] Refactor ZeRO-1 #5145

alanwaketan merged 5 commits into pytorch:master from hgt312:refactor_zero1

JackCaoG requested a review from

alanwaketan 3 years ago

hgt312 force pushed from 792a47ea to 354bda8c 3 years ago

hgt312 force pushed from 354bda8c to 31b1cc8e 3 years ago

hgt312 force pushed from 31b1cc8e to 4f6f4c11 3 years ago

hgt312 force pushed from 4f6f4c11 to 0c5503cc 3 years ago

hgt312 force pushed from 0c5503cc to 81e007fe 3 years ago

hgt312 force pushed from 92352647 to d1ba9e26 3 years ago

hgt312 force pushed from d1ba9e26 to 83dbd68e 3 years ago

hgt312 force pushed from 83dbd68e to 9b96e935 3 years ago

hgt312 force pushed from 9b96e935 to 7ec54af5 3 years ago

hgt312 force pushed from 7ec54af5 to 6b9f8635 3 years ago

refactor

fac2fde1

fix

de0b813f

fix

99f4fd67

add padding

cf99118c

more robust save/load

288bf804

hgt312 force pushed from 6b9f8635 to 288bf804 2 years ago

alanwaketan approved these changes on 2023-08-22

alanwaketan merged 059f6b9d into master 2 years ago

Reviewers

alanwaketan

Assignees

No one assigned

Labels

None yet

Milestone

No milestone