llama : save and restore kv cache for single seq id #6341
ggerganov
merged
34 commits into
ggml-org:master
from
kaetemi:feature/save-restore-seq
llama : save and restore kv cache for single seq id
662aaea8
remove trailing whitespace
54628178
respond error in case there's no space in the kv cache
ab1c46a7
add kv seq save restore to test case
02a18406
add --slot-save-path arg to enable save restore and restrict save loc…
b8e8facb
Returning 0 for some cases, instead of asserting.
b182f8f6
cleanup error cases
a2b48b95
ggerganov
commented
on 2024-03-28
rename sequence state functions
c4443d7a
rename state get set functions
4d5356bb
phymbert
requested changes
on 2024-03-29
add previous function names back in with DEPRECATED notice
bbcbf47b
update doc
8b5ae299
adjust endpoints to preferred style
a71ec3db
fix restoring zero cell count
bf1d4932
handle seq rm return value
8ab1a172
unused param
0d221367
keep in the size check
29f18c29
fix return types
f2e41b32
add server test case for slot save restore
92c46810
cleanup
60f685ff
phymbert
requested changes
on 2024-03-30
add cake
d38eef46
cleanup style
ea717f77
add special
b509b8b3
removing a whole sequence never fails
129b6ffe
move sequence state file functionality from server to llama to match …
8af72118
phymbert
approved these changes
on 2024-03-31
catch exceptions on save as well
3d6fa5bd
error log messages
b3f6da3d
check types for stricter restore
be714a0f
update server doc
0ccfbf2f
ggerganov
approved these changes
on 2024-04-04
readme : update API changes date
205c44c2
Merge branch 'master' into feature/save-restore-seq
d9fd0d7e
ngxson
commented
on 2024-04-04
strict filename validation
f2a4777d
move include, reject bom as well
4a4f3993
also reject empty filename
2fbf0c34
reject whitespace and trailing dot
bf94e9f7
ngxson
approved these changes
on 2024-04-06
Login to write a write a comment.
Login via GitHub