DeepSpeed
support autoTP with weight only quantization in DS inference path
#4750

Open

support autoTP with weight only quantization in DS inference path #4750

ftian1 wants to merge 3 commits into deepspeedai:master from ftian1:master

ftian1 requested a review from

RezaYazdaniAminabadi 2 years ago

ftian1 requested a review from

jeffra 2 years ago

ftian1 requested a review from

mrwyattii 2 years ago

ftian1 requested a review from

awan-10 2 years ago

ftian1 requested a review from

cmikeh2 2 years ago

ftian1 requested a review from

arashb 2 years ago

ftian1 requested a review from

tjruwase 2 years ago

baodii commented on 2023-12-12

ftian1 force pushed from 7fc67306 to 46f7ef25 1 year ago

ftian1 requested a review from

hwchen2017 1 year ago

ftian1 requested a review from

tohtana 1 year ago

ftian1 requested a review from

loadams 1 year ago

support the wildcard * in the weight_only ds_config

b3edd2f7

support autoTP with weight quantization in DS inference path

9b947a77

fix typo in LmHeadLinearAllreduce initialization

46f7ef25

loadams removed review request from

arashb 1 year ago

loadams removed review request from

cmikeh2 1 year ago

loadams removed review request from

awan-10 1 year ago

loadams removed review request from

mrwyattii 1 year ago

loadams removed review request from

RezaYazdaniAminabadi 1 year ago

loadams assigned

loadams 1 year ago

Reviewers

baodii

jeffra

tjruwase

hwchen2017

tohtana

loadams

Assignees

loadams

Labels

None yet

Milestone

No milestone

DeepSpeed support autoTP with weight only quantization in DS inference path #4750 Open

support autoTP with weight only quantization in DS inference path #4750

DeepSpeed
support autoTP with weight only quantization in DS inference path
#4750

Open