llama.cpp
Compute buffer and KV-cache aware layer distribution for multi-GPU inference
#14484
Open
Open
Compute buffer and KV-cache aware layer distribution for multi-GPU inference #14484
borebot
wants to merge
2 commits into
ggml-org:master
from
borebot:kv-compute-buffer-cache-aware-allocation
Login to write a write a comment.
Login via GitHub