Base automatically changed from
gg/llama-kv-cache-v6
to
master
1 year ago
kv-cache : prepare for SWA
c011e4ed
kv-cache : initial iSWA implementation
85f5fc53
kv-cache : rework error recovery logic
b9ce306e
models : fix Phi-3 SWA parameters
a4aafa53
model : adjust Granite to rope factor changes
c7d81757
server : check if context can do shifts
554b4d03
iswa : for now, always enable shifts (experiment)
4a258ffe
kv-cache : simplify SWA logic
e743246b
kv-cache : apply defrag when we fail to find slots for the batch
63901253
llama : update docs about llama_decode
86c526a0
kv-cache : update warning logs when no space for the batch is available
00731579
llama : add llama_kv_self_seq_pos_min()
12ee6db8
kv-cache : keep track of partial SWA computes and print warnings
ca52e196
server : disallow use cases involving partial SWA context
84742efd
llama : add param to control SWA cache size
c699abcb
minor : clean-up
5db11109
Login to write a write a comment.
Login via GitHub