microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into olruwase/bf16-updates

tjruwase committed 4 years ago

Verified ff99cb25

Enable CG headers on ROCm (#1821)

rraminen committed 4 years ago

Verified 7bcb4fab

Merge branch 'master' into olruwase/bf16-updates

tjruwase committed 4 years ago

Verified 4a505ecd

01 adam optimizer (#1790)

EugeneLYC committed 4 years ago

Verified b80e5624

Clip gradients of last stage tied weights

tjruwase committed 4 years ago

77b649d1

Link bf16 & fp32 params

tjruwase committed 4 years ago

19198688

[docker] simplify and update rocm dockerfile (#1819)

jeffra committed 4 years ago

Verified ac71a1a4

Remove optimizer restriction for bf16

tjruwase committed 4 years ago

b7d64fd7

Use O(n) instead of O(n^2)

tjruwase committed 4 years ago

ab61edb0

Merge branch 'olruwase/reduce_specified_gradients' of github.com:microsoft/DeepSpeed into olruwase/bf16-updates

tjruwase committed 4 years ago

89e054d8

Reduce specified gradients

tjruwase committed 4 years ago

de3616ca

Merge branch 'master' into olruwase/bf16-updates

jeffra committed 4 years ago

Verified 46cc2ce3

update gemfile (#1817)

jeffra committed 4 years ago

Verified 097efeb7

Merge branch 'master' into olruwase/bf16-updates

jeffra committed 4 years ago

Verified 3ee61cdb

jeffra committed 4 years ago

013df6a9

force amd install via sudo (#1815)

jeffra committed 4 years ago

Verified a32e9b33

Invoke hipify from op builder for JIT extension builds (#1807)

jithunnair-amd committed 4 years ago

Verified 350d74ca

add GitHub URL for PyPi (#1812)

andriyor committed 4 years ago

Verified d7684f4e

Fix grad norm for moe

tjruwase committed 4 years ago

61d51fd6

Rebase with master

tjruwase committed 4 years ago

8fbd4bfd

Reduce tied weight gradients

tjruwase committed 4 years ago

37011a92

Refactor moe/non-moe gradient reduction (#1811)

tjruwase committed 4 years ago

Verified 9f7126fc

Fix clip_grad key error

tjruwase committed 4 years ago

35ea3808

Enable loading fp16_zero_1 into bf16_zero_1 engine and vice versa

tjruwase committed 4 years ago

b81d862f

Synchronize the GPUs for the text-generation inference test (#1805)

RezaYazdaniAminabadi committed 4 years ago

Verified 60fc06c6

AMD support (#1430)

jeffra committed 4 years ago

Verified c3c8d5dd

Minjiaz/mos doc (#1802)

awan-10 committed 4 years ago

Verified f0304bd1

Fix grad norm scaling

tjruwase committed 4 years ago

0ad7c7d3

Unscale grad norm

tjruwase committed 4 years ago

f4977024

bf16 gradient clipping fix

tjruwase committed 4 years ago

27e5b956

Newer Older