DeepSpeed
Refactor MoE and Groups API to simplify model creation and mangement
#1798
Merged

Commits
  • add docs/proposal.
    awan-10 committed 4 years ago
  • save pending code.
    awan-10 committed 4 years ago
  • add more changes.
    awan-10 committed 4 years ago
  • Add changes to groups.
    awan-10 committed 4 years ago
  • use proper world group instead of None.
    awan-10 committed 4 years ago
  • fix the world group by cloning it
    awan-10 committed 4 years ago
  • fix group creation and return.
    awan-10 committed 4 years ago
  • fix typo
    awan-10 committed 4 years ago
  • another stupid typo.
    awan-10 committed 4 years ago
  • fix failed unit tests.
    awan-10 committed 4 years ago
  • use the correct method to get num_experts
    awan-10 committed 4 years ago
  • cleanup the layer code.
    awan-10 committed 4 years ago
  • remove the old code
    awan-10 committed 4 years ago
  • Move group creation to enginer. Fix several small things. Use helper functions.
    awan-10 committed 4 years ago
  • fix
    awan-10 committed 4 years ago
  • Merge branch 'master' into amawa/moe-api-updates
    awan-10 committed 4 years ago
  • Zheweiyao/moe post release residual moe (#289) (#290)
    awan-10 committed 4 years ago
  • feedback from Jeff/ammar review
    awan-10 committed 4 years ago
  • Fix unit tests and add checks to create groups.
    awan-10 committed 4 years ago
  • fix failing moe tests.
    awan-10 committed 4 years ago
  • fix name typo
    awan-10 committed 4 years ago
  • pass ep_size down to the layer.
    awan-10 committed 4 years ago
  • return rank
    awan-10 committed 4 years ago
  • All test pass. Fix pr-moe test
    awan-10 committed 4 years ago
  • add missing import
    awan-10 committed 4 years ago
  • parallelism spec based group creation.
    awan-10 committed 4 years ago
  • fix missing underscores.
    awan-10 committed 4 years ago
  • Fix module search.
    awan-10 committed 4 years ago
  • remove print.
    awan-10 committed 4 years ago
  • add missing _
    awan-10 committed 4 years ago
  • fix typos, fix split parameters for moe optimizer
    yaozhewei committed 4 years ago
  • update moe/cifar-moe tutorial.
    awan-10 committed 4 years ago
  • explain/fix use_residual.
    awan-10 committed 4 years ago
  • fix unit tests and remove groups from tutorial. mark todo for inference tutorial.
    awan-10 committed 4 years ago
  • get the moe flag from the right place on the transformer layer
    Reza Yazdani committed 4 years ago
  • Add in more feedback.
    awan-10 committed 4 years ago
  • Add in feedback
    awan-10 committed 4 years ago
  • Merge branch 'master' into amawa/moe-api-updates
    awan-10 committed 4 years ago
Loading