DeepSpeed
Fixed bug with hybrid engine generation when inference_tp_size > 1
#4493

Open

Fixed bug with hybrid engine generation when inference_tp_size > 1 #4493

hxdtest wants to merge 3 commits into deepspeedai:master from hxdtest:fix_attention_mask

gather attention mask

1a597655

hxdtest requested a review from

jeffra 1 year ago

hxdtest requested a review from

tjruwase 1 year ago

hxdtest changed the title ~~gather attention mask~~ Fixed bug with hybrid engine generation when inference_tp_size > 1 1 year ago

Merge branch 'master' into fix_attention_mask

3c7dc490

Merge branch 'master' into fix_attention_mask

e9eb917a

tjruwase requested a review from

lekurile 1 year ago

Reviewers

jeffra

tjruwase

lekurile

Assignees

No one assigned

Labels

None yet

Milestone

No milestone