llama.cpp
CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows
#29948
Merged
Go
Login via GitHub
Home
Pricing
FAQ
Install
Login
via GitHub
Overview
Commits
14
Changes
View On
GitHub
CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows
#29948
ggerganov
merged 14 commits into
ggml-org:master
from
anujj:cuda-mmq-fused-gate-up-interleaved
github-actions
added
testing
github-actions
added
ggml
github-actions
added
CUDA
anujj
marked this pull request as ready for review
43 days ago
anujj
requested a review
from
ggerganov
43 days ago
anujj
requested a review
43 days ago
JohannesGaessler
commented on 2026-10-08
anujj
force pushed
to
b6e0f033
42 days ago
JohannesGaessler
commented on 2026-10-17
cuda: fuse FFN gate/up and GLU in MMQ with interleaved gate/up rows
b1d280ef
cuda: assert the fused MMQ gate argument and trim redundant comments
6e59326f
cuda: move the fused MMQ hint check to should_fuse and add TODOs
525f87c1
cuda: pass prec_src1 to the fused MMQ write-back
744d4035
cuda: fuse only with efficient tiling and no fallback, merge the GLU …
4d20b62d
cuda: require has_fusion explicitly, drop the fusion-only src1 precis…
ebadd88c
cuda: launch the fused MMQ kernel through the stream-k kernel with on…
a11b8801
cuda: inline the MMQ tiling-efficiency check
30840769
tests: one fused gate/up MMQ case per type and GLU op
352d5106
cuda: always launch the fused MMQ kernel with one block per tile
7d9a32d4
tests: shorten the fused gate/up MMQ test comment
6e34e56d
anujj
force pushed
to
6e34e56d
30 days ago
JohannesGaessler
commented on 2026-10-23
cuda: select the fused MMQ loader and write-back in ggml_cuda_mmq_get…
5d4638a5
cuda: run the fused MMQ kernel on the plain tiling path
8a357f4f
cuda: put the column tiles innermost in the MMQ plain tiling path
d63ccd4f
JohannesGaessler
approved these changes on 2026-10-24
JohannesGaessler
added
merge ready
ggerganov
merged
c49edf09
into master
26 days ago
Login to write a write a comment.
Login via GitHub
Reviewers
JohannesGaessler
ggerganov
Assignees
No one assigned
Labels
testing
ggml
merge ready
CUDA
Milestone
No milestone
Login to write a write a comment.
Login via GitHub