DeepSpeed
let allgather and alltoall execute in parallel when both attention and MOE used TP
#7723

Open

let allgather and alltoall execute in parallel when both attention and MOE used TP #7723

taozhiwei wants to merge 2 commits into deepspeedai:master from taozhiwei:myfeature2

taozhiwei requested a review from

tohtana 91 days ago

taozhiwei requested a review from

tjruwase 91 days ago

taozhiwei requested a review from

loadams 91 days ago

taozhiwei force pushed from fc49a60a to 14be0fd5 91 days ago

taozhiwei force pushed from 14be0fd5 to bccd2bc2 91 days ago

let allgather and alltoall execute in parallel when both attention an…

de3ea101

taozhiwei force pushed from bccd2bc2 to de3ea101 91 days ago

Merge branch 'master' into myfeature2

b4e84f13

Reviewers

tohtana

tjruwase

loadams

Assignees

No one assigned

Labels

None yet

Milestone

No milestone