microsoft/DeepSpeed

Pull Requests Commits

Merge branch 'master' into neox-q-int8

RezaYazdaniAminabadi committed 2 years ago

Verified 77735015

fix Tensor contiguous bug in model_compression (#2671)

xiaoxiawu-microsoft committed 2 years ago

Verified be6d19f0

[fp16] lower initial_scale_power (#2663)

stas00 committed 2 years ago

Verified f30a0308

[Bug Fixed] use torch.cuda.is_available() (#2661)

wkcn committed 2 years ago

Verified 323c266c

Remove unnecessary device synchronization for stage 2 (#2500)

li-yi-dong committed 2 years ago

Verified 97deaaec

Abstract accelerator (step 2) (#2560)

delock committed 2 years ago

Verified 9548d48f

Fix Opt injection (#2541)

RezaYazdaniAminabadi committed 2 years ago

Verified 95d9a1b6

Merge branch 'master' into neox-q-int8

RezaYazdaniAminabadi committed 2 years ago

Verified 847c85f0

fix some issue with GPT-J and OPT ckpt-loading

Reza Yazdani committed 2 years ago

5658141b

[launcher] fail gracefully if hostname -i doesn't work as expected (#2631)

jeffra committed 2 years ago

Verified a091bc22

[doc] fix `min_loss_scale` default (#2660)

stas00 committed 2 years ago

Verified f2ea7a38

reduce the ckpt-loading code

Reza Yazdani committed 2 years ago

85ab3d10

fix the transposition when using int8

Reza Yazdani committed 2 years ago

0d201876

fix merging checkpoint for qkv params

Reza Yazdani committed 2 years ago

b8d731d4

Merge branch 'master' into neox-q-int8

RezaYazdaniAminabadi committed 2 years ago

Verified 376b854b

fix some issues with the replace policies

Reza Yazdani committed 2 years ago

88a8b149

tweaks to ds-attn, distilbert policy, and mup (#2649)

jeffra committed 2 years ago

Verified d9b788d7

support int8 for neox

Reza Yazdani committed 2 years ago

d5513e9d

fix assertion error in zero stage 3 (#2647)

GuanhuaWang committed 2 years ago

Verified 6375cb3f

Fix issue w. bloom when changing tp size (#2645)

jeffra committed 2 years ago

Verified e0aa84c5

[inference] ds-attention refactor w.r.t. ops (#2623)

jeffra committed 2 years ago

Verified bb68c526

[zero-3] Handle forward parameter return correctly in nested cases (#2642)

samyam committed 2 years ago

Verified a298a43a

Fix typo in autotuner.py (#2639)

eltociear committed 2 years ago

Verified 6273dffc

add enable_each_rank_log to deepspeed/launcher/runner.py (#2571)

inkcherry committed 2 years ago

Verified 11f5daba

Unit tests setup own venv (#2628)

mrwyattii committed 2 years ago

Verified 6fff50f5

[launcher] parse hostfile via regex and added error checks (#2626)

jeffra committed 2 years ago

Verified 8c56c25d

[inference] check for unsupported model generate args (#2627)

jeffra committed 2 years ago

Verified 5676f5ec

Add Megatron CI workflow (#2614)

mrwyattii committed 2 years ago

Verified df985fac

Update AVX512 Detection (#2621)

cmikeh2 committed 2 years ago

Verified a25c31b6

fixes #2498 (#2603)

Alexander Jipa committed 2 years ago

Verified 0f0e38c5

Older