microsoft/DeepSpeed

Pull Requests Commits

fix some issue with saving the variance for normalize forward kernel

Reza Yazdani committed 3 years ago

135ffe30

Merge branch 'master' into transformer/fix-layer-norm

jeffra committed 3 years ago

Verified af34cc47

[CI] install fix for HF tests and use dict instead of json for some tests (#1405)

jeffra committed 3 years ago

Verified 9e5c0c5c

remove unused grad keys

Reza Yazdani committed 3 years ago

5eae60f0

Modify normalize kernels to save variance in floating-point for backward

Reza Yazdani committed 3 years ago

f26900ec

[CI] Add HF transformers tests (#958)

awan-10 committed 3 years ago

Verified c1829c45

fix the workspace allocation for the transformer kernel

Reza Yazdani committed 3 years ago

4017a7c9

Fix from Felipe and Young for loading checkpoints. (#1389)

awan-10 committed 3 years ago

Verified 86dd6a64

[zero_to_fp32] fix padding removal (#1380)

stas00 committed 3 years ago

Verified 364994ad

add tutorial on pytorch profiler usage (#1350)

cli99 committed 3 years ago

Verified 51a2e916

Sparse attn triton v1.0 support + torch1.8 test runner (#1374)

jeffra committed 3 years ago

Verified 6996bb01

fix: support three digit layer numbers (#1377)

adammoody committed 3 years ago

Verified 4ad8019c

jeffra committed 3 years ago

45a498d0

[zero_to_fp32] adapt to 4-bytes alignment in z2 (#1372)

stas00 committed 3 years ago

Verified 30537e71

[zero Init] fix regression (#1373)

stas00 committed 3 years ago

Verified cf22a691

Introduce a device rank when setting device (#1370)

Sean Naren committed 3 years ago

Verified 90398a70

jeffra committed 3 years ago

a708b187

Update setup.py (#1361)

anukaal committed 3 years ago

Verified 8e577c92

fix unit test failure when merging PP ckpt files (#1359)

eltonzheng committed 3 years ago

Verified 9b915fed

Remove dropout as client code can do it independently. (#1354)

awan-10 committed 3 years ago

Verified 9f5939d2

Correctness fix PP+ZeRO for gradient accumulation (#1264)

jeffra committed 3 years ago

Verified b712babe

Update website hiring link

jeffra committed 3 years ago

Verified bff6126f

update hiring link

jeffra committed 3 years ago

Verified 168fce30

Update matmul.py (#1349)

hyunwoongko committed 3 years ago

Verified eb97a424

Added 4-byte alignment on NCCL/RCCL (#1328)

amathews-amd committed 3 years ago

Verified 2c628439

[actions] revert unit-test build name

jeffra committed 3 years ago

Verified 3e7d06ad

[actions] add torch version runner label

jeffra committed 3 years ago

Verified 0a32c3e2

[actions] add master to formatting trigger

jeffra committed 3 years ago

Verified 86b948f9

[actions] update branch triggers

jeffra committed 3 years ago

Verified 8f299be5

[actions] split formatting and unit tests into two jobs (#1339)

jeffra committed 3 years ago

Verified 600db091

Older