llama.cpp
llama : KV cache view API + better KV cache management
#4170

Merged

llama : KV cache view API + better KV cache management #4170

ggerganov merged 4 commits into master from kv-cache-opts

llama : keep track of used KV cells + better KV cache management

79cb8f00

ggerganov added need feedback

ggerganov requested a review from

KerfuffleV2 1 year ago

llama : zero KV cache used upon clear

671f639c

KerfuffleV2 approved these changes on 2023-11-23

llama : allow exporting a view of the KV cache (#4180)

5df7d06c

ggerganov changed the title ~~llama : keep track of used KV cells + better KV cache management~~ llama : KV cache view API + better KV cache management 1 year ago

KerfuffleV2 commented on 2023-11-23

common : add -dkvc arg for enabling kv cache dumps

f8e9f114

ggerganov merged 6b0a7420 into master 1 year ago

Reviewers

KerfuffleV2

Assignees

No one assigned

Labels

need feedback

Milestone

No milestone

llama.cpp llama : KV cache view API + better KV cache management #4170 Merged

llama : KV cache view API + better KV cache management #4170

llama.cpp
llama : KV cache view API + better KV cache management
#4170

Merged