llama.cpp
CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover
#26079

Merged

CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover #26079

ORippler merged 7 commits into ggml-org:master from praneshgo:pgonegandla/mmvq-mmq-crossover

github-actions added ggml

github-actions added CUDA

gaugarg-nv requested changes on 2026-07-27

ORippler commented on 2026-07-27

praneshgo requested a review from

gaugarg-nv 31 days ago

CUDA: runtime GGML_CUDA_MMVQ_MAX to tune the mvq->MMQ decode crossover

ff05098b

Added Blackwell specific switch point, to reduce dependence on runtim…

f2a3efae

github-actions added documentation

github-actions added model

github-actions added build

github-actions added testing

github-actions added Vulkan

github-actions added devops

github-actions added server

github-actions added SYCL

github-actions added Apple Metal

github-actions added OpenCL

github-actions added Hexagon

github-actions added mtmd

github-actions added WebGPU

github-actions added server/ui

github-actions added conversion

github-actions added vendor

praneshgo force pushed 29 days ago

praneshgo changed the title ~~CUDA: runtime GGML_CUDA_MMVQ_MAX to tune the mvq->MMQ decode crossover~~ CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover 29 days ago

praneshgo requested a review from

ORippler 29 days ago

gaugarg-nv removed model

gaugarg-nv removed build

gaugarg-nv removed testing

gaugarg-nv removed Vulkan

gaugarg-nv removed devops

gaugarg-nv removed server

gaugarg-nv removed SYCL

gaugarg-nv removed Apple Metal

gaugarg-nv removed OpenCL

gaugarg-nv removed Hexagon

gaugarg-nv removed mtmd

gaugarg-nv removed WebGPU

gaugarg-nv removed server/ui

gaugarg-nv removed conversion

gaugarg-nv removed vendor

gaugarg-nv removed documentation

Add per-HW switch point values for DGX Spark and removing runtime env…

f2aace7f

praneshgo force pushed to f2aace7f 28 days ago

Adding switch points for Ada, tested on RTX 4090

db1508d8

gaugarg-nv commented on 2026-08-03

Modifying DGX Spark numbers based on latest run and adding some comme…

9052a5ce

praneshgo requested a review from

gaugarg-nv 24 days ago

praneshgo marked this pull request as ready for review 24 days ago

praneshgo requested a review 24 days ago

gaugarg-nv approved these changes on 2026-08-10

ORippler requested changes on 2026-08-18

praneshgo requested a review from

ORippler 9 days ago

ORippler requested changes on 2026-08-19

Reverting an unnecessary conditional

614d5244

praneshgo requested a review from

ORippler 8 days ago

ORippler approved these changes on 2026-08-19

Update ggml/src/ggml-cuda/mmvq.cu

41a70d7e

gaugarg-nv approved these changes on 2026-08-20

ORippler merged 2b562109 into master 7 days ago

Reviewers

gaugarg-nv

ORippler

Assignees

No one assigned

Labels

ggml CUDA

Milestone

No milestone

llama.cpp CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover #26079 Merged

CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover #26079

llama.cpp
CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover
#26079

Merged