PR #4280 llama : pad KV cache size

llama : pad KV cache size #4280

ggerganov merged 2 commits into master from gg/pad-kv-cache

llama : pad KV cache size to 32

3e68df86

ggerganov force pushed from 75ba5ba6 to 3e68df86 1 year ago

metal : try to improve batched decoding

3cb1c348

ggerganov force pushed from 10cb4593 to 3cb1c348 1 year ago

ggerganov merged d7b800b8 into master 1 year ago

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone