PR #4657 Add n_key_dim and n_value_dim

Add n_key_dim and n_value_dim #4657

ggerganov merged 7 commits into ggml-org:master from n_value_dim

ggerganov requested changes on 2023-12-29

Add n_key_dim and n_value_dim

94d170b7

Fix `llm_build_kqv` to use `n_value_gqa`

e96fad12

Rebase

6200da58

Rename variables

51e251a8

Fix llm_build_kqv to be more generic wrt n_embd_head_k

f56cbce6

ggerganov requested changes on 2023-12-31

Update default values for n_embd_head_k and n_embd_head_v

522e5349

Fix llm_load_tensors: the asserts were not backcompat

4c1c0d68

ggerganov approved these changes on 2024-01-02

ggerganov merged 83e633c2 into master 1 year ago

postmasters deleted the n_value_dim branch 1 year ago

Reviewers

ggerganov

Assignees

No one assigned

Labels

None yet

Milestone

No milestone