llama.cpp
CUDA: Add `fastdiv` to `k_bin_bcast*`, giving 1-3% E2E performance
#15872

Merged

CUDA: Add `fastdiv` to `k_bin_bcast*`, giving 1-3% E2E performance #15872

JohannesGaessler merged 4 commits into ggml-org:master from ORippler:osimons/add_fastdiv_to_k_bin_bcast

Add fastdiv and fastmodulo to k_bin_bcast kernel

956a1d06

github-actions added Nvidia GPU

github-actions added ggml

JohannesGaessler commented on 2025-09-08

Address review comments

b63af608

ORippler changed the title ~~CUDA: Add `fastdiv` and `fastmodulo` to `k_bin_bcast*`, giving 1-3% E2E performance~~ CUDA: Add `fastdiv` to `k_bin_bcast*`, giving 1-3% E2E performance 7 days ago

`prod_` instead of `prod` suffix

3cd67088

Add test case for `k_bin_bcast_unravel` in CUDA backend

4014ae38

ORippler requested a review from

JohannesGaessler 6 days ago

github-actions added testing

JohannesGaessler approved these changes on 2025-09-10

JohannesGaessler merged 00681dfc into master 6 days ago

ORippler deleted the osimons/add_fastdiv_to_k_bin_bcast branch 6 days ago

Reviewers

JohannesGaessler

Assignees

No one assigned

Labels

testing Nvidia GPU ggml

Milestone

No milestone

llama.cpp CUDA: Add `fastdiv` to `k_bin_bcast*`, giving 1-3% E2E performance #15872 Merged

CUDA: Add `fastdiv` to `k_bin_bcast*`, giving 1-3% E2E performance #15872

llama.cpp
CUDA: Add `fastdiv` to `k_bin_bcast*`, giving 1-3% E2E performance
#15872

Merged