microsoft/DeepSpeed

Pull Requests Commits

Remove mutable default parameter in init_inference() (#2540)

aphedges committed 2 years ago

Verified 4abf637f

Add MII tests (#2533)

mrwyattii committed 2 years ago

Verified c5ee27f7

Make DS-Inference config readable from JSON (#2537)

mrwyattii committed 2 years ago

Verified 8b4318b9

Ensure is initialized for SD (#2534)

cmikeh2 committed 2 years ago

Verified 57e0a550

Fixes for torch 1.14 due to new torch.numel return type (#2522)

jeffra committed 2 years ago

Verified 21105521

awan-10 committed 2 years ago

803bf2a4

rename the vars in quant config.

awan-10 committed 2 years ago

92f7aab4

Merge branch 'master' into quantize-inference

RezaYazdaniAminabadi committed 2 years ago

Verified 741e80e4

fix quantization parameters

Reza Yazdani Aminabadi committed 2 years ago

bc1d63ec

Reza Yazdani Aminabadi committed 2 years ago

efab0aa0

Initial dequant library implementation (#2521)

cmikeh2 committed 2 years ago

Verified 30c8d8a8

Merge branch 'quantize-inference' of github.com:microsoft/DeepSpeed into quantize-inference

Reza Yazdani committed 2 years ago

44184ca9

Reza Yazdani committed 2 years ago

dd03ae68

Merge branch 'master' into quantize-inference

RezaYazdaniAminabadi committed 2 years ago

Verified 410faf74

Update codeowners (#2525)

jeffra committed 2 years ago

Verified 0b265326

Add more config params for quantization

Reza Yazdani Aminabadi committed 2 years ago

e2f6fe91

fix a few issues to run inference on 2 A100-80G (4 A100-40G)

Reza Yazdani committed 2 years ago

53418114

add q-int4 on top of q-int8

Reza Yazdani Aminabadi committed 2 years ago

d2997bf4

Reza Yazdani committed 2 years ago

2ce22d7d

add configurable quantization for enabling 4-bit inference

Reza Yazdani Aminabadi committed 2 years ago

4114beac

Add note about nvcc/hipcc requirement (#2519)

jeffra committed 2 years ago

Verified 7ce371b1

Add missing Inference sub-configs (#2518)

mrwyattii committed 2 years ago

Verified fe678544

Fix backward compatibility for InferenceConfig (#2516)

mrwyattii committed 2 years ago

Verified e59f8054

Deepspeed quantization library v0.1 (#2450)

lokoppakmsft committed 2 years ago

Verified 78d4ca1f

Add max_tokens alias to max_out_tokens arg to maintain backwards compatibility (#2508)

lekurile committed 2 years ago

Verified d40a15fc

Update docs to autogenerate pydantic config model docs (#2509)

mrwyattii committed 2 years ago

Verified 43bf035c

DeepSpeed inference config. (#2459) (#2472)

awan-10 committed 2 years ago

Verified b5d18a6a

jeffra committed 2 years ago

a4ceabb6

Fix typos: deepseed -> deepspeed (#2499)

jinyouzhi committed 2 years ago

Verified 06e00f61

Make data contiguous before the inplace reshape-copy_ function (#2489)

lokoppakmsft committed 2 years ago

Verified f2710bbe

Newer Older