DeepSpeed
ZeRO stage 1 refresh
#1042
Merged
Overview
Commits
12
Changes
View On GitHub

ZeRO stage 1 refresh #1042

jeffra merged 12 commits into master from jeffra/z1-refresh-4
jeffra
jeffra add zero-1 mode to zero-2 codebase
e3444631
jeffra fixes to support PP
a2fb18aa
jeffra jeffra requested a review from arashashari arashashari 4 years ago
jeffra jeffra requested a review from awan-10 awan-10 4 years ago
jeffra jeffra requested a review from cli99 cli99 4 years ago
jeffra jeffra requested a review from conglongli conglongli 4 years ago
jeffra jeffra requested a review from eltonzheng eltonzheng 4 years ago
jeffra jeffra requested a review from minjiaz minjiaz 4 years ago
jeffra jeffra requested a review from niumanar niumanar 4 years ago
jeffra jeffra requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 4 years ago
jeffra jeffra requested a review from samyam samyam 4 years ago
jeffra jeffra requested a review from ShadenSmith ShadenSmith 4 years ago
jeffra jeffra requested a review from tjruwase tjruwase 4 years ago
tjruwase
tjruwase approved these changes on 2021-05-05
jeffra update json docs to add grad_hooks param
10eda146
jeffra Merge branch 'master' into jeffra/z1-refresh-4
7ba9c961
samyam
samyam commented on 2021-05-06
jeffra Merge branch 'master' into jeffra/z1-refresh-4
d994f88e
jeffra Merge branch 'master' into jeffra/z1-refresh-4
faf3b3a6
jeffra add missing comma in merge confict fix
df795cad
jeffra remove grad hooks, unify grad reduce from engine
a9ef5994
jeffra Merge branch 'master' into jeffra/z1-refresh-4
bcd36474
jeffra fixes to support PP
567339eb
jeffra Merge branch 'master' into jeffra/z1-refresh-4
4e6f92ba
samyam
samyam approved these changes on 2021-05-17
jeffra Merge branch 'master' into jeffra/z1-refresh-4
d11285f9
jeffra jeffra merged cfa63f5d into master 4 years ago
mrwyattii mrwyattii deleted the jeffra/z1-refresh-4 branch 2 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum