PR #4480 [Bug fix] Add rope_theta for llama config

[Bug fix] Add rope_theta for llama config #4480

mrwyattii merged 9 commits into deepspeedai:master from cupertank:fix-rope-theta

Add rope_theta for llama config

7b0b002b

cupertank requested a review from

RezaYazdaniAminabadi 2 years ago

cupertank requested a review from

jeffra 2 years ago

cupertank requested a review from

mrwyattii 2 years ago

cupertank requested a review from

awan-10 2 years ago

cupertank requested a review from

cmikeh2 2 years ago

cupertank requested a review from

arashb 2 years ago

Merge branch 'master' into fix-rope-theta

a22d4b40

mrwyattii assigned

mrwyattii 2 years ago

Add rope_theta to bias_add_transform_0213

f1b119ef

Merge branch 'master' into fix-rope-theta

a8e2933a

Merge branch 'master' into fix-rope-theta

364b9dfa

mrwyattii approved these changes on 2023-10-13

lekurile approved these changes on 2023-10-13

Fix CI problems

6e131a6c

Merge branch 'master' into fix-rope-theta

6cf06b45

Add rope_theta to linear layer

0e12d1d4

Merge branch 'master' into fix-rope-theta

a3a675a0

mrwyattii merged beed962c into master 2 years ago

cupertank deleted the fix-rope-theta branch 2 years ago

Reviewers

lekurile

mrwyattii

RezaYazdaniAminabadi

jeffra

awan-10

cmikeh2

arashb

Assignees

mrwyattii

Labels

None yet

Milestone

No milestone

DeepSpeed [Bug fix] Add rope_theta for llama config #4480 Merged

[Bug fix] Add rope_theta for llama config #4480

DeepSpeed
[Bug fix] Add rope_theta for llama config
#4480

Merged