DeepSpeed
Big science fix passing multiple tensors
#1400
Merged
Overview
Commits
45
Changes
View On GitHub
Commits
  • Minor tweaks to support Megatron 2.4 + DS 3D
    Shaden Smith committed 5 years ago
  • pipe partitioning
    Shaden Smith committed 5 years ago
  • re-enable grad buffer partitioning
    Shaden Smith committed 5 years ago
  • Avoid partitioning small activations
    tjruwase committed 5 years ago
  • Merge pull request #4 from ShadenSmith/olruwase/partition_activation
    tjruwase committed 5 years ago
  • send/recv
    Shaden Smith committed 5 years ago
  • isend/irecv missing wait
    Shaden Smith committed 5 years ago
  • turn off async ops
    Shaden Smith committed 5 years ago
  • Merge branch 'megatron2.4-3d-sendrecv' into megatron2.4-3d
    Shaden Smith committed 5 years ago
  • less verbose load
    Shaden Smith committed 5 years ago
  • Merge branch 'master' into megatron2.4-3d
    jeffra committed 5 years ago
  • added shaden's set_train_batch_size patches, plus formatting
    jeffra committed 5 years ago
  • Adds engine.was_step_applied() (#1251)
    GitHub committed 5 years ago
  • Cleaning up tensor/pipe parallel accounting. (#1252)
    GitHub committed 5 years ago
  • Correctness fix PP+ZeRO for gradient accumulation + updates from master (#1263)
    jeffra committed 5 years ago
  • dont clear grads in stage 1 code path
    jeffra committed 5 years ago
  • prevent none grads from being reduced
    jeffra committed 5 years ago
  • fix empty grad zero tests
    jeffra committed 5 years ago
  • Use mpu in DeepSpeedConfig() call (#1271)
    tjruwase committed 5 years ago
  • API for obtaining global gradient norm (#1292)
    tjruwase committed 5 years ago
  • turn excessive noise off (#1293)
    stas00 committed 5 years ago
  • [zero] restore fp16 params if no zero ckpts available (#1322)
    jeffra committed 5 years ago
  • Fix PP checkpoint bloat (#1324)
    tjruwase committed 5 years ago
  • update for cuda-11.4 (#1329)
    jeffra committed 5 years ago
  • Try something out
    thomasw21 committed 4 years ago
  • Woops
    thomasw21 committed 4 years ago
  • Make deepspeed pass any types of dtypes between stages
    thomasw21 committed 4 years ago
  • Woops
    thomasw21 committed 4 years ago
  • Woops 2
    thomasw21 committed 4 years ago
  • Woops 3
    thomasw21 committed 4 years ago
  • Try debugging deadlock
    thomasw21 committed 4 years ago
  • Fix dtype
    thomasw21 committed 4 years ago
  • Fix some more things
    ThomasYoungson committed 4 years ago
  • Woops
    ThomasYoungson committed 4 years ago
  • Use list comprehension instead of for loops, and increase the number of asserts
    thomasw21 committed 4 years ago
  • Run pre-commit
    thomasw21 committed 4 years ago
  • Use ValueError + error msg instead of NotImplemetedError
    thomasw21 committed 4 years ago
  • Merge remote-tracking branch 'origin/master' into big-science-fix-passing-multiple-tensors
    thomasw21 committed 4 years ago
  • Use tuples instead of lists
    thomasw21 committed 4 years ago
  • Merge branch 'master' into big-science-fix-passing-multiple-tensors
    jeffra committed 4 years ago
  • Merge branch 'master' into big-science-fix-passing-multiple-tensors
    tjruwase committed 4 years ago
  • Make sure to set as input a tensor when required, instead of a tuple of tensors
    thomasw21 committed 4 years ago
  • Merge branch 'big-science-fix-passing-multiple-tensors' of github.com:thomasw21/DeepSpeed into big-science-fix-passing-multiple-tensors
    thomasw21 committed 4 years ago
  • Update inputs as well
    thomasw21 committed 4 years ago
  • Merge branch 'master' into big-science-fix-passing-multiple-tensors
    tjruwase committed 4 years ago
FAQ Terms Privacy Refunds Impressum
Loading