PR #1985 DeepSpeed Comm. Backend v1

Staging comms v1 (#301)

Quentin-Anthony committed 3 years ago

Delete stage1.py

awan-10 committed 3 years ago

Delete distributed.py

awan-10 committed 3 years ago

revert deepspeed/__init__.py logging calls

Quentin-Anthony committed 3 years ago

Delete test.py

Quentin-Anthony committed 3 years ago

Update comments and move custom comm ops to internal functions

Quentin-Anthony committed 3 years ago

Merge branch 'staging-comms-next' of https://github.com/microsoft/DeepSpeed into staging-comms-next

Quentin-Anthony committed 3 years ago

Remove unnecessary print and update backend description

Quentin-Anthony committed 3 years ago

Relax assertion to allow Megatron-DeepSpeed MoE to use ZeRO 1

Quentin-Anthony committed 3 years ago

Simplify ZeRO stage 1 check for previous commit

Quentin-Anthony committed 3 years ago

Remove misleading world_size prints

Quentin-Anthony committed 3 years ago

Remove custom backend references

Quentin-Anthony committed 3 years ago

Merge branch 'master' into staging-comms-next

jeffra committed 3 years ago

Delete nccl.cpp

Quentin-Anthony committed 3 years ago

Delete alltoall.py

Quentin-Anthony committed 3 years ago

Delete runall.py

Quentin-Anthony committed 3 years ago

Delete all_reduce.py

Quentin-Anthony committed 3 years ago

Merge branch 'master' into staging-comms-next

jeffra committed 3 years ago

Add isend/irecv comm ops

Quentin-Anthony committed 3 years ago

Add pre-commit check and clean up references to torch.dist in comments

Quentin-Anthony committed 3 years ago

Updated pre-commit error

Quentin-Anthony committed 3 years ago

Clean up comm.py logic

Quentin-Anthony committed 3 years ago

Added comm constants file to avoid raw string assignment

Quentin-Anthony committed 3 years ago

Revert "Relax assertion to allow Megatron-DeepSpeed MoE to use ZeRO 1"

Quentin-Anthony committed 3 years ago

DeepSpeed DeepSpeed Comm. Backend v1 #1985 Merged

DeepSpeed
DeepSpeed Comm. Backend v1
#1985

Merged