DeepSpeed
ZeRO3, improved parameter all-gather operation
#1188
Merged
Overview
Commits
39
Changes
View On GitHub
Commits
  • remove norm(), avoid memcpy after allgather
    zarzen committed 5 years ago
  • WIP: wrapped ncclAllgather as customized op in DS
    zarzen committed 5 years ago
  • WIP: integrated into partition_parameters
    zarzen committed 5 years ago
  • Fix format
    zarzen committed 5 years ago
  • Merge branch 'master' into impr_allgather_params
    zarzen committed 5 years ago
  • cleaned dead code, modified unit test
    zarzen committed 5 years ago
  • Merge branch 'master' into impr_allgather_params
    zarzen committed 5 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • removed customized c++ extension
    zarzen committed 4 years ago
  • Merge remote-tracking branch 'origin/master' into impr_allgather_params
    zarzen committed 4 years ago
  • change torch.ones to torch empty
    zarzen committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • typo
    zarzen committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • warn if not cuda tensor for allgather
    zarzen committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    jeffra committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • fix formatting
    zarzen committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • fix: move ds_tensor to cuda device
    zarzen committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • remove try clause on the path for fetching params
    zarzen committed 4 years ago
  • Merge branch 'microsoft:master' into impr_allgather_params
    zarzen committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
  • Merge branch 'master' into impr_allgather_params
    tjruwase committed 4 years ago
FAQ Terms Privacy Refunds Impressum
Loading