microsoft/DeepSpeed

Pull Requests Commits

add flash attention

Reza Yazdani committed 3 years ago

ea246363

Merge branch 'master' of github.com:microsoft/DeepSpeed

Reza Yazdani committed 3 years ago

41257346

MOE residual matmult unit test (#2323)

samadejacobs committed 3 years ago

Verified 80b10d0c

Merge branch 'master' of github.com:microsoft/DeepSpeed

Reza Yazdani committed 3 years ago

b55a26eb

jeffra committed 3 years ago

0f0a7a5d

Add more options to inference benchmark (#2325)

mrwyattii committed 3 years ago

Verified 15923810

only override forward if using cuda-graph (#2291)

jeffra committed 3 years ago

Verified cf638be9

add quant unit test (#2315)

GuanhuaWang committed 3 years ago

Verified 95d11517

refactor to use mem_access (#2317)

mrwyattii committed 3 years ago

Verified c199edac

ZeRO-Inference blog - Update README (#2322)

tjruwase committed 3 years ago

Verified 060078ab

ZeRO-Inference blog - wrap up (#2321)

tjruwase committed 3 years ago

Verified f5230be8

ZeRO-Inference blog (#2271)

tjruwase committed 3 years ago

Verified 276eec7b

Upgrade P40 tests to torch 1.8 (#2316)

mrwyattii committed 3 years ago

Verified 18ee381e

add inference eval scripts (#2303)

jeffra committed 3 years ago

Verified 9595dff6

Add unit tests for residual_add kernels (#2307)

arashb committed 3 years ago

Verified 9d541a63

Fix the residual add mp scaling for GPTNeoX (#2310)

arashb committed 3 years ago

Verified efa8aded

Add tensor parallel inference unit tests (#2232)

mrwyattii committed 3 years ago

Verified a691ec60

Update relu.cu with mem_access_utils (#2306)

molly-smith committed 3 years ago

Verified d0dfe38d

Unit test for bias add kernel (#2298)

mrwyattii committed 3 years ago

Verified b2d550ab

Fix for Zero3 when MP>1 and at least one batch param undefined (#2289)

Quentin-Anthony committed 3 years ago

Verified 4524b8db

fix for pytest picking up local deepspeed dir instead of installed deepspeed (#2299)

mrwyattii committed 3 years ago

Verified 7e085b62

fused bias relu unittest (#2297)

molly-smith committed 3 years ago

Verified 5102c4ab

Offload all gradients to nvme (#2282)

tjruwase committed 3 years ago

Verified c8641032

[ds-inference] fix progress bar (#2286)

stas00 committed 3 years ago

Verified b146aa35

Refactor universal checkpointing and tensor fragments (#2253)

tjruwase committed 3 years ago

Verified 53182531

Fp32 accuracy bug fix (#2285)

RezaYazdaniAminabadi committed 3 years ago

Verified 47e030f5

Memory Access Utility (#2276)

cmikeh2 committed 3 years ago

Verified c84bca37

Merge branch 'master' of github.com:microsoft/DeepSpeed

Reza Yazdani committed 3 years ago

bdff8bf9

Add to codeowners file (#2279)

cmikeh2 committed 3 years ago

Verified 5505e247

add a new unit test for cuda ops (#2278)

awan-10 committed 3 years ago

Verified 91eba15a

Older