PR #1985 DeepSpeed Comm. Backend v1

DeepSpeed Comm. Backend v1 #1985

jeffra merged 24 commits into master from staging-comms-next

Staging comms v1 (#301)

867a8537

awan-10 requested a review from

jeffra 3 years ago

awan-10 requested a review from

samyam 3 years ago

awan-10 requested a review from

tjruwase 3 years ago

awan-10 requested a review from

ShadenSmith 3 years ago

awan-10 requested a review from

conglongli 3 years ago

awan-10 requested a review from

cli99 3 years ago

awan-10 requested a review from

eltonzheng 3 years ago

awan-10 requested a review from

minjiaz 3 years ago

awan-10 requested a review from

RezaYazdaniAminabadi 3 years ago

awan-10 commented on 2022-05-27

Delete stage1.py

c93fcfef

Delete distributed.py

7f8ca013

awan-10 commented on 2022-05-27

revert deepspeed/__init__.py logging calls

977ee324

Delete test.py

68eb9f4e

Update comments and move custom comm ops to internal functions

54796bb8

Merge branch 'staging-comms-next' of https://github.com/microsoft/Dee…

c06c72d3

Remove unnecessary print and update backend description

f070a0c8

Relax assertion to allow Megatron-DeepSpeed MoE to use ZeRO 1

9976681a

Simplify ZeRO stage 1 check for previous commit

09063a3d

Remove misleading world_size prints

656b4152

jeffra commented on 2022-06-01

Remove custom backend references

859c5f83

jeffra changed the base branch from staging-comms-next-v2 to master 3 years ago

Merge branch 'master' into staging-comms-next

4d710b4f

Quentin-Anthony changed the title ~~Staging comms v1 (#301)~~ DeepSpeed Comm. Backend v1 (#301) 3 years ago

jeffra changed the title ~~DeepSpeed Comm. Backend v1 (#301)~~ DeepSpeed Comm. Backend v1 3 years ago

Delete nccl.cpp

108064ed

Delete alltoall.py

ce5001f7

Delete runall.py

73afaa8f

Delete all_reduce.py

e51021f6

Merge branch 'master' into staging-comms-next

6678344d

jeffra approved these changes on 2022-06-07

Add isend/irecv comm ops

604eae78

Add pre-commit check and clean up references to torch.dist in comments

2dd23f3a

Updated pre-commit error

b9027440

tjruwase commented on 2022-06-09

Clean up comm.py logic

3c5e065e

deepspeedai deleted a comment from rocm-mici on 2022-06-09

Added comm constants file to avoid raw string assignment

971c349e

Revert "Relax assertion to allow Megatron-DeepSpeed MoE to use ZeRO 1"

5e0c8c19

tjruwase approved these changes on 2022-06-09

jeffra merged 36ad3119 into master 3 years ago

jeffra deleted the staging-comms-next branch 3 years ago

jeffra restored the head branch 3 years ago

mrwyattii deleted the staging-comms-next branch 2 years ago

Reviewers

tjruwase

jeffra

Quentin-Anthony

samyam

ShadenSmith

conglongli

cli99

eltonzheng

minjiaz

RezaYazdaniAminabadi

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

DeepSpeed DeepSpeed Comm. Backend v1 #1985 Merged

DeepSpeed Comm. Backend v1 #1985

DeepSpeed
DeepSpeed Comm. Backend v1
#1985

Merged