llama.cpp
CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints
#26048

Merged

CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints #26048

ORippler merged 7 commits into ggml-org:master from kmorennv:prefil_fuse_epilogue_nvfp4

github-actions added testing

github-actions added ggml

github-actions added CUDA

gaugarg-nv commented on 2026-07-27

ORippler commented on 2026-07-27

github-actions added build

kmorennv marked this pull request as ready for review 94 days ago

kmorennv requested a review from

ggerganov 94 days ago

kmorennv requested a review 94 days ago

ORippler dismissed these changes on 2026-08-14

kmorennv force pushed 57 days ago

kmorennv closed this 57 days ago

kmorennv reopened this 57 days ago

aendk commented on 2026-09-23

kmorennv marked this pull request as draft 43 days ago

kmorennv force pushed to 8361cdb0 38 days ago

kmorennv marked this pull request as ready for review 37 days ago

kmorennv force pushed from 8361cdb0 to ca9dd41b 19 days ago

kmorennv force pushed from ca9dd41b to d84e0650 10 days ago

ORippler commented on 2026-11-04

ORippler dismissed their stale review 10 days ago

Scope was clarified/unrelated changes removed

kmorennv force pushed from d84e0650 to 9d2f0f31 5 days ago

ORippler commented on 2026-11-11

ggml-cuda: Apply NVFP4 weight scale in MMQ write-back

8dc8a2b9

ggml-cuda: Fuse mul_mat(_id) + scale into MMQ

cbc5bf36

ggml-cuda: Tests for the fused scale

f3319523

ggml-cuda: reuse existing graph to test fusion in mmq epiloque

5d72c3eb

ggml-cuda: rename test_mul_mat_vec_fusion to test_mul_mat_fusion for …

9e718f7b

ggml-cuda: refine comments for clarity on MMQ dispatch and fusion che…

5821eefa

kmorennv force pushed from 8ff27f14 to 5821eefa 3 days ago

ggml-cuda: refactor fusion checks for mul_mat operations and remove u…

7aeea7c2

ORippler approved these changes on 2026-11-11

ORippler requested a review from

am17an 3 days ago

am17an approved these changes on 2026-11-11

ORippler merged 0d69255c into master 2 days ago

Reviewers

am17an

ORippler

aendk

gaugarg-nv

ggerganov

Assignees

No one assigned

Labels

build testing ggml CUDA

Milestone

No milestone

llama.cpp CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints #26048 Merged

CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints #26048

llama.cpp
CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints
#26048

Merged