moe-cache : LRFU/MQ-like hybrid cache, ClassLRU #31233
ggerganov
merged
19 commits into
ggml-org:master
from
sanmai:feat/moe-cache-classlru
moe-cache : LRFU/MQ-inspired hybrid cache
6a8bf8ed
sanmai
commented
on 2026-10-27
make the clock advance by N tokens: use the real token count (ne[1] i…
5dd386c3
sel is a RESHAPE node, the orig input is in src[0], so sel->src[0]->n…
9ac703c3
sanmai
commented
on 2026-10-29
sanmai
commented
on 2026-10-29
sanmai
changed the title
c for the class index, last_use for last use time
9f4c30df
explain the roles of these
c09c6f58
compute the key once
e05f14e8
use the key directly instead of re-reading
b7c08c51
show how the score is computed from the age
231bc3e4
remove the branch and clarify exp2 and log2 relationship
98ce01e2
reword the header comment and elaborate on unused slots and sorting
cfea435b
not a copy, not just skip
82466a9a
std::array maybe
7649b1f1
Revert "std::array maybe"
1ba27ea1
half life it is
5e2fe317
add key and iteration helpers
27979671
extract decayed_crf
a00d1bca
am17an
approved these changes
on 2026-10-30
bump classes to 8, document the formula, display the cache size in to…
8be980ff
assert the computed number of classes matches the half life
aff64547
am17an
commented
on 2026-11-03
Merge branch 'master' into feat/moe-cache-classlru
515064f5
am17an
approved these changes
on 2026-11-06
Login to write a write a comment.
Login via GitHub