llama.cpp
Improve utilization (and tg t/s) by reducing K_QUANTS_PER_ITERATION to 1 on DMMV path
#25063

Merged

Improve utilization (and tg t/s) by reducing K_QUANTS_PER_ITERATION to 1 on DMMV path #25063

ggerganov merged 4 commits into ggml-org:master from aicss-genai:reduce-k-quants-per-iteration

sycl: add supported types to ggml_sycl_supports_reorder_dmmv

0c4df1ca

sycl: set K_QUANTS_PER_ITERATION=1 to improve utilization

0fab1f3e

sycl: replace QK_WARP_SIZE with WARP_SIZE for QK_5

af503101

sycl: add missing types to ggml_backend_sycl_buffer_init_tensor

47a97548

malsbat requested a review 50 days ago

github-actions added ggml

github-actions added SYCL

arthw approved these changes on 2026-06-29

arthw added merge ready

ggerganov merged d2090861 into master 39 days ago

Reviewers

arthw

Assignees

No one assigned

Labels

ggml merge ready SYCL

Milestone

No milestone