microsoft/DeepSpeed

Pull Requests Commits

skip quantization for tensor smaller than 500k

HeyangQin committed 2 years ago

bd2519c1

Merge remote-tracking branch 'origin/master' into HeyangQin/mixed_precision_lora

HeyangQin committed 2 years ago

aa4f28a2

replace get_accelerator().current_device()

HeyangQin committed 2 years ago

c25cf6b0

Merge branch 'HeyangQin/mixed_precision_lora' of https://github.com/microsoft/DeepSpeed into HeyangQin/mixed_precision_lora

HeyangQin committed 2 years ago

e0e8cf49

HeyangQin committed 2 years ago

3470949c

HeyangQin committed 2 years ago

b1cb5973

add zero++ paper link (#3974)

jeffra committed 2 years ago

Verified cbf2f61a

jeffra committed 2 years ago

5b2dc7a8

fix(cpu_accelerator): :bug: Convert LOCAL_SIZE to integer (#3971)

javsalgar committed 2 years ago

Verified f5c834a6

Create accelerator for apple silicon GPU Acceleration (#3907)

NripeshN committed 2 years ago

Verified 31ac29dd

do bcast only pp_group_size>1 (#3915)

inkcherry committed 2 years ago

Verified 05a6cee1

Use device_name instead of device index to support other device (#3933)

hipudding committed 2 years ago

Verified 7528035c

fix Megatron-DeepSpeed links (#3956)

conglongli committed 2 years ago

Verified 4d965416

Merge branch 'master' into HeyangQin/mixed_precision_lora

HeyangQin committed 2 years ago

Verified ce96d9a0

add mixed precision lora tutorial

HeyangQin committed 2 years ago

94cf3c4a

Fix docs for checkpoints (#3955)

loadams committed 2 years ago

Verified ed34ddca

lazy load of quantizer kernels

HeyangQin committed 2 years ago

a06c5644

Merge branch 'HeyangQin/mixed_precision_lora' of https://github.com/microsoft/DeepSpeed into HeyangQin/mixed_precision_lora

HeyangQin committed 2 years ago

7b2b6a4b

remove unused async_op

HeyangQin committed 2 years ago

b3ad4253

Merge branch 'master' into HeyangQin/mixed_precision_lora

HeyangQin committed 2 years ago

Verified cabf59c0

fix "ERROR: failed to solve: nvidia/cuda:11.7.0-devel-ubuntu18.04: docker.io/nvidia/cuda:11.7.0-devel-ubuntu18.04: not found" (#3930)

KaiChen1008 committed 2 years ago

Verified 45cecc05

HeyangQin committed 2 years ago

8fe8c87d

skip quantization for small parameters

HeyangQin committed 2 years ago

9d531688

add xTrimoPGLM (#3940)

jeffra committed 2 years ago

Verified aa54dba0

Update zero_to_fp32.py (#3936)

PicoCreator committed 2 years ago

Verified 103884ae

Reduce Unit Test Times (Part 3) (#3850)

mrwyattii committed 2 years ago

Verified aef6c65c

remove the call to param.ds_tensor from print (#3928)

HeyangQin committed 2 years ago

Verified e59f69a8

make sure dequantization is finished

HeyangQin committed 2 years ago

ec1f154c

Del comment deepspeed.zero.Init() can be used as a decorator (#3894)

hipudding committed 2 years ago

Verified e292343d

fix: change ==NONE to is (#3923)

digger-yu committed 2 years ago

Verified ce535945

Older