DeepSpeed
[zero3] release tmp memory when consolidating fp16 weights
#1220
Merged
Overview
Commits
5
Changes
View On GitHub

[zero3] release tmp memory when consolidating fp16 weights #1220

jeffra merged 5 commits into deepspeedai:master from stas00:less-mem-fp16-save
stas00
stas00 release tmp memory when consolidating fp16 weights
908fd562
stas00 stas00 requested a review from awan-10 awan-10 4 years ago
stas00 stas00 requested a review from cli99 cli99 4 years ago
stas00 stas00 requested a review from conglongli conglongli 4 years ago
stas00 stas00 requested a review from eltonzheng eltonzheng 4 years ago
stas00 stas00 requested a review from jeffra jeffra 4 years ago
stas00 stas00 requested a review from minjiaz minjiaz 4 years ago
stas00 stas00 requested a review from niumanar niumanar 4 years ago
stas00 stas00 requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 4 years ago
stas00 stas00 requested a review from samyam samyam 4 years ago
stas00 stas00 requested a review from ShadenSmith ShadenSmith 4 years ago
stas00 stas00 requested a review from tjruwase tjruwase 4 years ago
stas00 document the need to use modifier_rank=0 when wanting memory release
27031b48
stas00 clarify
0c8e6c86
stas00 clarify
6e89a083
tjruwase Merge branch 'master' into less-mem-fp16-save
5518faf0
tjruwase
tjruwase approved these changes on 2021-07-12
jeffra jeffra merged 2660cc4d into master 4 years ago
stas00 stas00 deleted the less-mem-fp16-save branch 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum