DeepSpeed
ZeRO 2+3 memory estimators
#965
Merged
Overview
Commits
12
Changes
View On GitHub

ZeRO 2+3 memory estimators #965

jeffra merged 12 commits into deepspeedai:master from stas00:mem-estimation-utils
stas00
stas00 memory estimators
d94bde1c
stas00 stas00 requested a review from arashashari arashashari 5 years ago
stas00 stas00 requested a review from awan-10 awan-10 5 years ago
stas00 stas00 requested a review from cli99 cli99 5 years ago
stas00 stas00 requested a review from conglongli conglongli 5 years ago
stas00 stas00 requested a review from eltonzheng eltonzheng 5 years ago
stas00 stas00 requested a review from jeffra jeffra 5 years ago
stas00 stas00 requested a review from minjiaz minjiaz 5 years ago
stas00 stas00 requested a review from niumanar niumanar 5 years ago
stas00 stas00 requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 5 years ago
stas00 stas00 requested a review from samyam samyam 5 years ago
stas00 stas00 requested a review from ShadenSmith ShadenSmith 5 years ago
stas00 stas00 requested a review from tjruwase tjruwase 5 years ago
stas00 wip
d6c2336a
stas00 better in GBs
9d4dd5a5
samyam
samyam commented on 2021-04-16
samyam
stas00
stas00
stas00 rework the code
1d323cc9
stas00
stas00 Merge remote-tracking branch 'origin/master' into mem-estimation-utils
80f89323
stas00 add zero2 estimators and full docs
956b2d2e
stas00 stas00 changed the title [wip] zero memory estimators ZeRO 2+3 memory estimators 5 years ago
stas00 style
f0c7c16c
stas00
stas00 cleanup
bdda5610
stas00 add API; add examples
ad4b1440
samyam
samyam approved these changes on 2021-06-04
samyam Merge branch 'master' into mem-estimation-utils
0dd406d4
stas00 more intuitive naming
5336cd0f
jeffra Merge branch 'master' into mem-estimation-utils
e0766598
jeffra jeffra merged 0c1802cc into master 5 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum