DeepSpeed
Cleaning up tensor/pipe parallel accounting.
#1252
Merged
Overview
Commits
4
Changes
View On GitHub

Cleaning up tensor/pipe parallel accounting. #1252

ShadenSmith merged 4 commits into deepspeedai:big-science from ShadenSmith:shaden/pipe_replicated
ShadenSmith
removes repeated overflow log
f76e59ee
pipe_replicated
467ebf22
_pipe_replicated -> ds_pipe_replicated
d11244a2
ShadenSmith ShadenSmith requested a review from jeffra jeffra 4 years ago
ShadenSmith ShadenSmith requested a review from awan-10 awan-10 4 years ago
ShadenSmith ShadenSmith requested a review from cli99 cli99 4 years ago
ShadenSmith ShadenSmith requested a review from conglongli conglongli 4 years ago
ShadenSmith ShadenSmith requested a review from eltonzheng eltonzheng 4 years ago
ShadenSmith ShadenSmith requested a review from minjiaz minjiaz 4 years ago
ShadenSmith ShadenSmith requested a review from niumanar niumanar 4 years ago
ShadenSmith ShadenSmith requested a review from RezaYazdaniAminabadi RezaYazdaniAminabadi 4 years ago
ShadenSmith ShadenSmith requested a review from samyam samyam 4 years ago
ShadenSmith ShadenSmith requested a review from tjruwase tjruwase 4 years ago
jeffra
jeffra approved these changes on 2021-07-26
ShadenSmith
ShadenSmith commented on 2021-07-26
Adds send/recv fallback to bcast when torch version <= 1.8
f66caaf9
ShadenSmith ShadenSmith merged d6945dea into big-science 4 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum