llama.cpp
ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory
#26141

Merged

ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory #26141

am17an merged 1 commit into ggml-org:master from KakaruHayate:fix-mmq-smem-fallback

KakaruHayate requested a review 48 days ago

github-actions added ggml

github-actions added CUDA

KakaruHayate changed the title ~~CUDA/MUSA: fall back to cuB when no MMQ tile size fits in smem~~ CUDA/MUSA: fall back to cuBlass when no MMQ tile size fits in smem 47 days ago

KakaruHayate changed the title ~~CUDA/MUSA: fall back to cuBlass when no MMQ tile size fits in smem~~ ggml-cuda : fall back to cuBLAS when no MMQ tile size fits in shared memory 47 days ago

KakaruHayate force pushed 47 days ago

KakaruHayate force pushed 45 days ago

KakaruHayate changed the title ~~ggml-cuda : fall back to cuBLAS when no MMQ tile size fits in shared memory~~ ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory 45 days ago

ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory

0e27a00c

KakaruHayate marked this pull request as draft 45 days ago

KakaruHayate force pushed to 0e27a00c 45 days ago

KakaruHayate marked this pull request as ready for review 45 days ago

am17an approved these changes on 2026-07-29

ORippler approved these changes on 2026-07-29

am17an merged caa596ab into master 45 days ago

Reviewers

ORippler

am17an

Assignees

No one assigned

Labels

ggml CUDA

Milestone

No milestone

llama.cpp ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory #26141 Merged

ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory #26141

llama.cpp
ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory
#26141

Merged