microsoft/DeepSpeed

Pull Requests Commits

Testing for hidden dim alignment

samyam committed 4 years ago

5692c62b

also removing alignment from flat fp16 buffers

samyam committed 4 years ago

9cd813d2

Adding static_scale test back for Z3, and also changing hidden size to be not divisile by world_size

samyam committed 4 years ago

bf53561c

samyam committed 4 years ago

ac9266ee

Merge branch 'master' into fix-misaligned-grad

Shaden Smith committed 4 years ago

Verified a55a8dd4

Fix mis-aligned-grad

samyam committed 4 years ago

Verified 2ffaecbb

bumping DSE pointer (#847)

Shaden Smith committed 4 years ago

Verified 564eb4bd

Fix regression in runner (#843)

jeffra committed 4 years ago

Verified 2e6692c8

replace home env with ~

jeffra committed 4 years ago

Verified 49496364

Model scale changing 5x to 3x

samyam committed 4 years ago

Verified 6adc19a6

jeffra committed 4 years ago

Verified af548971

bump DSE to include ZeRO-3

jeffra committed 4 years ago

9c5eee3d

Fix zero3 tutorial link

jeffra committed 4 years ago

Verified 75ffdaf7

update tutorial/doc links for zero3 (#835)

jeffra committed 4 years ago

Verified d7de9165

ZeRO 3 Offload (#834)

samyam committed 4 years ago

Verified 599258f9

Update ZeRO-Offload tutorials (#824)

tjruwase committed 4 years ago

Verified ba33e86e

Fixing gelu_checkpointing memory issue (#812)

RezaYazdaniAminabadi committed 4 years ago

Verified 8295d7a8

issue with the implementation of column_sum_reduce (#804)

zmxdream committed 4 years ago

Verified 937c5cee

fixed typo (#802)

vfdev-5 committed 4 years ago

Verified db987cf1

document the requirement to call for all ranks (#801)

stas00 committed 4 years ago

Verified 7eb083c2

fixing the compiling issue for the AMD architecture (#796)

RezaYazdaniAminabadi committed 4 years ago

Verified 490e6f7c

Delete out2 (#798)

vfdev-5 committed 4 years ago

Verified 62396b71

Fix the bias-add and add the layer-norm-eps parameter (#791)

RezaYazdaniAminabadi committed 4 years ago

Verified e2dfcadf

Fixing the module-inject Api (#786)

RezaYazdaniAminabadi committed 4 years ago

Verified 48065c06

[doc] fix incorrect param name (#773)

stas00 committed 4 years ago

Verified e60e92eb

Update engine.py (#767)

jeffra committed 4 years ago

Verified 29fa4b2b

CPU-Adam fix for scalar mode (#735)

RezaYazdaniAminabadi committed 4 years ago

Verified ee1ffe2e

Fix transformer kernel CUDA illegal memory access error (#765)

conglongli committed 4 years ago

Verified 1fcc5f7a

[dist] set args.local_rank to LOCAL_RANK (#764)

jeffra committed 4 years ago

Verified 68e138b6

Fix NameError: name 'dist' is not defined (#763)

tma15 committed 4 years ago

Verified 8067efa4

Older