DeepSpeed
support autoTP with weight only quantization in DS inference path
#4750

Open

support autoTP with weight only quantization in DS inference path #4750

ftian1 wants to merge 3 commits into deepspeedai:master from ftian1:master

ftian1 requested a review from

RezaYazdaniAminabadi 2 years ago

ftian1 requested a review from

jeffra 2 years ago

ftian1 requested a review from

mrwyattii 2 years ago

ftian1 requested a review from

awan-10 2 years ago

ftian1 requested a review from

cmikeh2 2 years ago

ftian1 requested a review from

arashb 2 years ago

ftian1 requested a review from

tjruwase 2 years ago

baodii commented on 2023-12-12

ftian1 force pushed from 7fc67306 to 46f7ef25 344 days ago

ftian1 requested a review from

hwchen2017 344 days ago

ftian1 requested a review from

tohtana 344 days ago

ftian1 requested a review from

loadams 344 days ago

support the wildcard * in the weight_only ds_config

b3edd2f7

support autoTP with weight quantization in DS inference path

9b947a77

fix typo in LmHeadLinearAllreduce initialization

46f7ef25

loadams removed review request from

arashb 344 days ago

loadams removed review request from

cmikeh2 344 days ago

loadams removed review request from

awan-10 344 days ago

loadams removed review request from

mrwyattii 344 days ago

loadams removed review request from

RezaYazdaniAminabadi 344 days ago

loadams assigned

loadams 344 days ago

Reviewers

baodii

jeffra

tjruwase

hwchen2017

tohtana

loadams

Assignees

loadams

Labels

None yet

Milestone

No milestone

DeepSpeed support autoTP with weight only quantization in DS inference path #4750 Open

support autoTP with weight only quantization in DS inference path #4750

DeepSpeed
support autoTP with weight only quantization in DS inference path
#4750

Open