MP ZeRO++ #3954
HeyangQin
merged
51 commits into
master
from
HeyangQin/mixed_precision_lora
zero++ tutorial PR (#3783)
df1859d6
[Fix] _conv_flops_compute when padding is a str and stride=1 (#3169)
d81a6ad6
fix interpolate flops compute (#3782)
a8c182a4
use `Flops Profiler` to test `model.generate()` (#2515)
c4c442f0
revert PR #3611 (#3786)
fc9e1ee0
bump to 0.9.6
40045dc7
ZeRO++ chinese blog (#3793)
49a0a1bb
remove staging trigger (#3792)
2c62cb4c
DeepSpeed-Triton for Inference (#3748)
4dc65f7b
ZeRO++ (#3784)
e1119d8b
adding zero++ to navigation panel of deepspeed.ai (#3796)
01b843aa
Add ZeRO++ Japanese blog (#3797)
319b64ed
Bug Fixes for autotuner and flops profiler (#1880)
b4a2c0af
Missing strided copy for gated MLP (#3788)
b7e1010b
Requires grad checking. (#3789)
e5b1eadb
bump to 0.10.0
9c756cf8
Fix Bug in transform.cu (#3534)
a204edc7
bug fix: triton importing error (#3799)
f6e2e38b
Merge branch 'master' of github.com:microsoft/DeepSpeed
c1a7d3cc
Merge branch 'master' of github.com:microsoft/DeepSpeed
65ed5483
Merge branch 'master' of github.com:microsoft/DeepSpeed
d7ac3296
Merge branch 'master' of github.com:microsoft/DeepSpeed
83f1102e
Merge branch 'master' of github.com:microsoft/DeepSpeed
16555b27
Merge branch 'master' of github.com:microsoft/DeepSpeed
9d7b654a
init commit for mixed precision lora
2efb73dd
fix format
1147885c
patch _allgather_params & minor fixes
1bec51f5
make sure initial quantization are finished
5b3c460a
make sure dequantization is finished
ec1f154c
skip quantization for small parameters
9d531688
fix format
8fe8c87d
Merge branch 'master' into HeyangQin/mixed_precision_lora
cabf59c0
remove unused async_op
b3ad4253
Merge branch 'HeyangQin/mixed_precision_lora' of https://github.com/m…
7b2b6a4b
lazy load of quantizer kernels
a06c5644
add mixed precision lora tutorial
94cf3c4a
Merge branch 'master' into HeyangQin/mixed_precision_lora
ce96d9a0
cleanup mics
b1cb5973
cleanup mics
3470949c
Merge branch 'HeyangQin/mixed_precision_lora' of https://github.com/m…
e0e8cf49
replace get_accelerator().current_device()
c25cf6b0
Merge remote-tracking branch 'origin/master' into HeyangQin/mixed_pre…
aa4f28a2
Merge remote-tracking branch 'origin/master' into HeyangQin/mixed_pre…
f7cb5493
add kwargs to mics
d5013092
fix format
b5a41fab
seperate code and tutorial
74c27605
Merge branch 'master' into HeyangQin/mixed_precision_lora
9f68cdad
awan-10
approved these changes
on 2023-08-18
Merge branch 'master' into HeyangQin/mixed_precision_lora
f8020116
Merge branch 'master' into HeyangQin/mixed_precision_lora
a6bd4544
Merge branch 'master' into HeyangQin/mixed_precision_lora
3d527b24
fix _all_gather in zero3
9e277ba8
Login to write a write a comment.
Login via GitHub