llama.cpp
llama : refactor k-shift implementation + initial defragmentation
#5691
Merged
Merged
llama : refactor k-shift implementation + initial defragmentation #5691
ggerganov
merged
25 commits into
master
from
gg/refactor-k-shift
llama : refactor k-shift implementation
1a999819
llama : rename llama_kv_cache_seq_shift to llama_kv_cache_seq_add
dd392191
llama : cont k-shift refactoring + normalize type names
89b2a43c
minor : fix MPI builds
2b9a9bff
llama : reuse n_rot from the build context
5f5b1b57
llama : revert enum name changes from this PR
42ddf484
llama : update llama_rope_type
31e1ec92
llama : add comment about rope values
decea312
llama : fix build
8f9fe6dd
passkey : apply kv cache updates explicitly
79e27617
llama : change name to llama_kv_cache_update()
18da970e
llama : add llama_kv_cache_seq_pos_max()
b75ec64e
passkey : fix llama_kv_cache_seq_pos_max() usage
032ff857
llama : some llama_kv_cell simplifications
715a3433
llama : add llama_kv_cache_compress (EXPERIMENTAL)
fdfa5bc7
Merge branch 'master' into gg/refactor-k-shift
0d6f8734
llama : add alternative KV cache merging (EXPERIMENTAL)
9ec749df
llama : add llama_kv_cache_defrag
65f21ec5
Merge branch 'master' into gg/refactor-k-shift
d141c749
llama : comments
1b6aeb83
llama : remove llama_kv_cache_compress
2d7203b9
ggerganov
changed the title
llama : defragment via non-overlapping moves
65323bc7
llama : ggml_graph based defrag implementation
4eaaace3
llama : switch the loop order in build_defrag
0b72ded5
llama : add comments
5a122c25
slaren
approved these changes
on 2024-02-25
Login to write a write a comment.
Login via GitHub