PR #3493 kv cache slot search improvements

kv cache slot search improvements #3493

KerfuffleV2 merged 3 commits into ggml-org:master from KerfuffleV2:feat-kv_cache_improvements

kv cache slot search improvements

abafd01e

KerfuffleV2 commented on 2023-10-05

KerfuffleV2 requested a review from

ggerganov 1 year ago

Use n_ctx in kv find slot for consistency

3144563d

Ensure kv cache head points to a valid slot in llama_decode internal

465b8f4f

ggerganov approved these changes on 2023-10-06

KerfuffleV2 merged 9ca79d5c into master 1 year ago

KerfuffleV2 deleted the feat-kv_cache_improvements branch 1 year ago

Reviewers

ggerganov

Assignees

No one assigned

Labels

None yet

Milestone

No milestone