llama.cpp
25ae3a9b - CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark (#26843)

Commit
18 hours ago
CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark (#26843) * CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark Signed-off-by: ynankani <ynankani@nvidia.com> * skip moe experts and allow others based on k geometry (allow only small idle tail) Signed-off-by: ynankani <ynankani@nvidia.com> * rename MMVQ DGX Spark params to GB10 and fix MSVC constexpr lambda capture Signed-off-by: ynankani <ynankani@nvidia.com> --------- Signed-off-by: ynankani <ynankani@nvidia.com>
Author
Parents
Loading