llama.cpp
0d69255c - CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints (#26048)

Commit
1 day ago
CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints (#26048) * ggml-cuda: Apply NVFP4 weight scale in MMQ write-back * ggml-cuda: Fuse mul_mat(_id) + scale into MMQ * ggml-cuda: Tests for the fused scale * ggml-cuda: reuse existing graph to test fusion in mmq epiloque * ggml-cuda: rename test_mul_mat_vec_fusion to test_mul_mat_fusion for consistency * ggml-cuda: refine comments for clarity on MMQ dispatch and fusion checks in mul_mat functions * ggml-cuda: refactor fusion checks for mul_mat operations and remove unused function
Author
Parents
Loading