DeepSpeed
[zero2] zero_param_shapes: switch to round_robin_fp16_groups
#1240
Merged
Overview
Commits
3
Changes
View On GitHub

[zero2] zero_param_shapes: switch to round_robin_fp16_groups #1240

tjruwase merged 3 commits into deepspeedai:master from stas00:z2-zero_to_fp32
stas00
stas00 zero_param_shapes: switch to round_robin_fp16_groups
9705cfed
stas00 stas00 requested a review from awan-10 awan-10 4 years ago
stas00 stas00 requested a review from cli99 cli99 4 years ago
stas00 stas00 requested a review from conglongli conglongli 4 years ago
stas00 stas00 requested a review from eltonzheng eltonzheng 4 years ago
stas00 stas00 requested a review from jeffra jeffra 4 years ago
stas00 stas00 requested a review from minjiaz minjiaz 4 years ago
stas00 stas00 requested a review from niumanar niumanar 4 years ago
stas00 stas00 requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 4 years ago
stas00 stas00 requested a review from samyam samyam 4 years ago
stas00 stas00 requested a review from ShadenSmith ShadenSmith 4 years ago
stas00 stas00 requested a review from tjruwase tjruwase 4 years ago
stas00 add test
32e9ecf4
stas00 stas00 changed the title [WIP] zero_param_shapes: switch to round_robin_fp16_groups [zero2] zero_param_shapes: switch to round_robin_fp16_groups 4 years ago
stas00 old torch workaround
489fe570
tjruwase
tjruwase approved these changes on 2021-07-20
tjruwase tjruwase merged 7392e459 into master 4 years ago
stas00 stas00 deleted the z2-zero_to_fp32 branch 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum