DeepSpeed
reduce all-to-all communication volume when both expert and non-expert are tensor-parallel
#5626
Merged
Merged
reduce all-to-all communication volume when both expert and non-expert are tensor-parallel #5626
loadams
merged
9 commits into
deepspeedai:master
from
myfeature
reduce all-to-all communication volume when both expert and non-exper…
6db21bf2
Merge branch 'master' into myfeature
9f98a521
tjruwase
approved these changes
on 2024-06-27
Merge branch 'master' into myfeature
f85527e4
Merge branch 'master' into myfeature
25e8e17b
Merge branch 'master' into myfeature
55aef6d8
Merge branch 'master' into myfeature
23b828f9
Merge branch 'master' into myfeature
1b84e356
Merge branch 'master' into myfeature
4a486aeb
Merge branch 'master' into myfeature
ada525b2
Login to write a write a comment.
Login via GitHub