DeepSpeed
fix num_kv_heads sharding in autoTP for the new in-repo Falcon-40B
#4654

Merged

fix num_kv_heads sharding in autoTP for the new in-repo Falcon-40B #4654

loadams merged 3 commits into deepspeedai:master from dc3671:master

fix num_kv_heads sharding in autoTP for the new in-repo Falcon-40B

70ef3152

dc3671 requested a review from

RezaYazdaniAminabadi 2 years ago

dc3671 requested a review from

jeffra 2 years ago

dc3671 requested a review from

mrwyattii 2 years ago

dc3671 requested a review from

awan-10 2 years ago

dc3671 requested a review from

cmikeh2 2 years ago

dc3671 requested a review from

arashb 2 years ago

Merge branch 'master' into master

594bca81

lekurile approved these changes on 2023-11-10

Merge branch 'master' into master

0ef8babb

loadams enabled auto-merge 2 years ago

loadams merged 6ea44d02 into master 2 years ago

Reviewers

lekurile

jeffra

mrwyattii

awan-10

cmikeh2

arashb

Assignees

No one assigned

Labels

None yet

Milestone

No milestone