microsoft/DeepSpeed

Pull Requests Commits

Update reduce ops and get_rank calls to use DS comms backend

Quentin-Anthony committed 3 years ago

9abeb4e3

Change remaining torch.dist calls to deepspeed.comm

Quentin-Anthony committed 3 years ago

0edf438b

Merge branch 'master' into staging-comms-v1

awan-10 committed 4 years ago

133a91e1

Undo parts of 8fe250ba282246e5f4b420bf2eeb7f2a44a8a996 and create new path to check for supported torch versions.

awan-10 committed 4 years ago

127dadaf

use torch.dist.group instead of local group object.

awan-10 committed 4 years ago

13216967

Fix torch versions less than 1.8 and move helpers to the end of the file.

awan-10 committed 4 years ago

8fe250ba

log_dist -> logger. Dist is not initialized before ds.init in new code so this will not work.

awan-10 committed 4 years ago

92bba2dc

PROD --> PRODUCT.

awan-10 committed 4 years ago

03c3ebbe

awan-10 committed 4 years ago

365fba0a

Deal with pp tests to use ds.comm.

awan-10 committed 4 years ago

06e8a7b1

syntax, format, and dist.init local imports.

awan-10 committed 4 years ago

5fcf3a11

replace all torch.dist with deepspeed.comm.

awan-10 committed 4 years ago

3882abd8

Create a PR to run tests. (#248)

awan-10 committed 4 years ago

Verified 83182f49

fix read-the-docs based on this issue: https://github.com/sphinx-doc/sphinx/issues/9727 (#1489)

jeffra committed 4 years ago

Verified 24dd285f

Proposal of how we might use sparse tensors for gradients (#1484)

Dipet committed 4 years ago

Verified fcb3ca5e

Add cpu adagrad (#1358)

fumihwh committed 4 years ago

Verified 8abdaee2

add moe+zero ckpt unit test. (#1429)

awan-10 committed 4 years ago

Verified 0b77d1d9

conglongli committed 4 years ago

Verified 29bee73f

fix pipeline engine (#1473)

conglongli committed 4 years ago

Verified 17a479dd

Ensure single zero3 context (#1462)

tjruwase committed 4 years ago

Verified 58a8e13c

Switch to use or not einsum op. Needed for ORT (#1456)

ganik committed 4 years ago

Verified 20bf1cc1

fix typos and add improvements (#1463)

stas00 committed 4 years ago

Verified dd224284

document about engine.load_checkpoint() (#1457)

skpig committed 4 years ago

Verified b4e5826a

Add basic MoE timing breakdown (#1428)

awan-10 committed 4 years ago

Verified 1fc74cb9

Fix docstrings for lr_schedules.py (#1455)

junxu committed 4 years ago

Verified c64a03db

fix sparse_attention imports (#1446)

olegklimov committed 4 years ago

Verified 80e263c5

webpage link edit (#1450)

Shaden Smith committed 4 years ago

Verified f88b0e5f

doc update (#1448)

jeffra committed 4 years ago

Verified 4e736237

[docs] 530b update (#1447)

jeffra committed 4 years ago

Verified 57a99542

fix cl for pp support (#1443)

conglongli committed 4 years ago

Verified cd7967d6

Older