transformers
`deepgemm_megamoe` EP implementation for DeepGEMM Mega MoE
#49606
Open
Open
`deepgemm_megamoe` EP implementation for DeepGEMM Mega MoE #49606
IlyasMoutawwakil
wants to merge
10 commits into
ep-masked-when-shared-batch
from
ep-passthrough-strategy
IlyasMoutawwakil
changed the title 
`passthrough` expert-parallel strategy, selected for DeepGEMM Mega MoE
a466c57c
Deprecate the Mega MoE router and experts parallel styles
6a1f6668
Remove the Mega MoE router and experts parallel styles
359335d2
`deepgemm_megamoe` strategy owns the Mega MoE symmetric buffers, shar…
a0038bd5
Allocate the Mega MoE buffer into the cache, then read it back
9d8c4d2c
Keep the Mega MoE forward in its original order
cd057eb7
Hand each rank's Mega MoE kernel a slice of the shared batch
7cb77529
Note that Mega MoE needs data-parallel inference to run as designed
ff15b657
Use EP implementation naming in Mega MoE
b6bbb4b7
Format EP implementation changes
6ca9df45
IlyasMoutawwakil
changed the title
Login to write a write a comment.
Login via GitHub