DeepSpeed
AutoEP: reorder expert rows with gathers on the collective path
#8671
Merged

Commits
  • Reorder AutoEP expert rows with gathers instead of a copy, a scatter and accumulating index_put
    yh0903 committed 54 days ago
  • Read the AutoEP row-gather index contiguously and state the permutation the reorder needs
    yh0903 committed 54 days ago
  • Test the AutoEP reorder in FP16, through strided rows and gradients, and in unpermute double backward
    yh0903 committed 54 days ago
  • Merge branch 'master' into yh0903/autoep-gather-permute
    pengdurice committed 51 days ago
  • Run AutoEP reorder tests in clean single-worker processes under pytest --forked
    yh0903 committed 51 days ago
Loading