llama.cpp
llama : KV cache view API + better KV cache management
#4170

Merged

Commits

llama : keep track of used KV cells + better KV cache management

ggerganov committed 2 years ago
llama : zero KV cache used upon clear

ggerganov committed 2 years ago
llama : allow exporting a view of the KV cache (#4180)

KerfuffleV2 committed 2 years ago
common : add -dkvc arg for enabling kv cache dumps

ggerganov committed 2 years ago