microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'staging-amd-v3' of github.com:microsoft/DeepSpeed into staging-amd-v3

jeffra committed 3 years ago

bec59b3a

add barrier between ckpt save/load

jeffra committed 3 years ago

e9f9896a

switch ds.init to use param dict instead of json file on disk

jeffra committed 3 years ago

deaac9d8

Use rtol to suppress spurious failure on AMD (#1387)

tjruwase committed 3 years ago

Verified 8484f246

turn back on old transformer tests

jeffra committed 3 years ago

7e39d6fb

add HALF_PRECISION_AVAILABLE macro

jeffra committed 3 years ago

b5feb4be

fix file-system race condition in saving/loading to disk on amd nodes

jeffra committed 3 years ago

7af87e32

[squash] Staging amd v2 (#1383)

jeffra committed 3 years ago

437d2c07

remove rocm skips and install 1bit on torch12

jeffra committed 3 years ago

8bf9ad7b

run unit tests with installed DS + fix bug in SA tests

jeffra committed 3 years ago

c2c7d74b

jeffra committed 3 years ago

5d92bce0

[squash] staging amd progress (#1353)

jeffra committed 3 years ago

ea7d4bfc

add tutorial on pytorch profiler usage (#1350)

cli99 committed 3 years ago

Verified 51a2e916

Sparse attn triton v1.0 support + torch1.8 test runner (#1374)

jeffra committed 3 years ago

Verified 6996bb01

fix: support three digit layer numbers (#1377)

adammoody committed 3 years ago

Verified 4ad8019c

jeffra committed 3 years ago

45a498d0

[zero_to_fp32] adapt to 4-bytes alignment in z2 (#1372)

stas00 committed 3 years ago

Verified 30537e71

[zero Init] fix regression (#1373)

stas00 committed 3 years ago

Verified cf22a691

Introduce a device rank when setting device (#1370)

Sean Naren committed 3 years ago

Verified 90398a70

jeffra committed 3 years ago

a708b187

Update setup.py (#1361)

anukaal committed 3 years ago

Verified 8e577c92

fix unit test failure when merging PP ckpt files (#1359)

eltonzheng committed 3 years ago

Verified 9b915fed

Remove dropout as client code can do it independently. (#1354)

awan-10 committed 3 years ago

Verified 9f5939d2

Correctness fix PP+ZeRO for gradient accumulation (#1264)

jeffra committed 3 years ago

Verified b712babe

Update website hiring link

jeffra committed 3 years ago

Verified bff6126f

update hiring link

jeffra committed 3 years ago

Verified 168fce30

Update matmul.py (#1349)

hyunwoongko committed 3 years ago

Verified eb97a424

Added 4-byte alignment on NCCL/RCCL (#1328)

amathews-amd committed 3 years ago

Verified 2c628439

[actions] revert unit-test build name

jeffra committed 3 years ago

Verified 3e7d06ad

[actions] add torch version runner label

jeffra committed 3 years ago

Verified 0a32c3e2

Older