DeepSpeed
Enable tensor fragments for zero 2 & 3
#2727
Merged

Commits
  • Enable tensor fragments for zero 2
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Update deepspeed/utils/tensor_fragment.py
    tjruwase committed 3 years ago
  • Update deepspeed/utils/tensor_fragment.py
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Support offload
    tjruwase committed 3 years ago
  • Support offload
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Support multi-gpu
    tjruwase committed 3 years ago
  • Cleanup
    tjruwase committed 3 years ago
  • WIP
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Update deepspeed/runtime/zero/stage3.py
    tjruwase committed 3 years ago
  • Master rebase
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Support padding
    tjruwase committed 3 years ago
  • Merge branch 'olruwase/tensor_fragments' of github.com:microsoft/DeepSpeed into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Update deepspeed/runtime/zero/stage3.py
    tjruwase committed 3 years ago
  • z3 optimizer state support; aligned api
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Support frozen z3 params
    tjruwase committed 3 years ago
  • Merge branch 'olruwase/tensor_fragments' of github.com:microsoft/DeepSpeed into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Unit tests
    tjruwase committed 3 years ago
  • Merge branch 'olruwase/tensor_fragments' of github.com:microsoft/DeepSpeed into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Check NVMe offload capability
    tjruwase committed 3 years ago
  • Merge branch 'olruwase/tensor_fragments' of github.com:microsoft/DeepSpeed into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Formatting
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Docs
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • More docs
    tjruwase committed 3 years ago
  • Merge branch 'olruwase/tensor_fragments' of github.com:microsoft/DeepSpeed into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • More docs
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Update docs/code-docs/source/zero3.rst
    tjruwase committed 3 years ago
  • More docs
    tjruwase committed 3 years ago
  • Update docs/code-docs/source/zero3.rst
    tjruwase committed 3 years ago
  • More docs
    tjruwase committed 3 years ago
  • More docs
    tjruwase committed 3 years ago
  • Update docs/code-docs/source/zero3.rst
    tjruwase committed 3 years ago
  • Update deepspeed/utils/tensor_fragment.py
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • More docs
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Support unsharded fp32 grad
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Remove debug prints
    tjruwase committed 3 years ago
  • Merge branch 'olruwase/tensor_fragments' of github.com:microsoft/DeepSpeed into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Fix off-by-one detection of empty grads
    tjruwase committed 3 years ago
  • Merge branch 'olruwase/tensor_fragments' of github.com:microsoft/DeepSpeed into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Update deepspeed/utils/tensor_fragment.py
    tjruwase committed 3 years ago
  • Update deepspeed/utils/tensor_fragment.py
    tjruwase committed 3 years ago
  • Update deepspeed/utils/tensor_fragment.py
    tjruwase committed 3 years ago
  • Update deepspeed/runtime/zero/stage3.py
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    mrwyattii committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Fix off-by-one error
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Skip ranks with no gradient data
    tjruwase committed 3 years ago
  • Formatting
    tjruwase committed 3 years ago
  • Merge branch 'olruwase/tensor_fragments' of github.com:microsoft/DeepSpeed into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Merge branch 'master' into olruwase/tensor_fragments
    tjruwase committed 3 years ago
  • Add license
    tjruwase committed 3 years ago
  • Fix license
    tjruwase committed 3 years ago
Loading