llama.cpp
CUDA: improve top-k algorithm selection
#28713
Merged

CUDA: improve top-k algorithm selection #28713

praneshgo
praneshgo CUDA: radix top-k for large row counts
a011ccd3
github-actions github-actions added ggml
github-actions github-actions added CUDA
ORippler
praneshgo
praneshgo praneshgo marked this pull request as ready for review 59 days ago
praneshgo praneshgo requested a review 59 days ago
ynankani
ynankani commented on 2026-09-22
albertnsoliz
ORippler
ORippler
ORippler ORippler closed this 53 days ago
anskumar01
praneshgo
praneshgo
ynankani ynankani reopened this 46 days ago
praneshgo praneshgo marked this pull request as draft 46 days ago
Merge branch 'master' into pgonegandla/topk-radix-only
04e682a2
CUDA: select the TOP_K implementation by shape
e4683d84
CUDA: make top-k comments less verbose
0d4ee054
praneshgo praneshgo changed the title CUDA: radix top-k for large row counts CUDA: improve top-k algorithm selection 45 days ago
github-actions github-actions added testing
praneshgo praneshgo marked this pull request as ready for review 44 days ago
praneshgo praneshgo requested a review from ggerganov ggerganov 44 days ago
praneshgo praneshgo requested a review from ynankani ynankani 44 days ago
praneshgo
gaugarg-nv
gaugarg-nv
gaugarg-nv requested changes on 2026-10-07
CUDA: remove the TOP_K width limit from supports_op
cb688931
CUDA: use DeviceTopK for single-row TOP_K if available
975bf810
praneshgo
praneshgo
praneshgo praneshgo requested a review from gaugarg-nv gaugarg-nv 43 days ago
ORippler
ORippler commented on 2026-10-07
ORippler
ORippler
CUDA: avoid ncols overflow in the TOP_K bitonic check
d2f31e40
CUDA: share the row chunking helper between argsort and top-k
036b9f4c
praneshgo
praneshgo praneshgo requested a review from ORippler ORippler 43 days ago
ORippler
ORippler requested changes on 2026-10-08
CUDA: do the TOP_K radix blocks_per_row math in int64_t
61c36829
CUDA: rename GGML_CUDA_TOP_K_NROWS_THRESHOLD_DEVICETOPK to GGML_CUDA_…
0454fc41
CUDA: share one sort helper between the bitonic and CUB TOP_K paths
8fb941ac
CUDA: update the TOP_K TODO, threshold and chunking comments
4bdba64a
tests: add TOP_K cases that span several row chunks
ebcdb647
praneshgo
gaugarg-nv
gaugarg-nv approved these changes on 2026-10-08
ORippler
ORippler commented on 2026-10-08
CUDA: use int64_t col in the TOP_K radix loops, fix threshold comment
a406e45b
am17an
am17an approved these changes on 2026-10-08
CUDA: limit TOP_K and ARGSORT support to ne[0] <= INT_MAX
45211ef7
praneshgo praneshgo requested a review from ORippler ORippler 42 days ago
ORippler
ORippler approved these changes on 2026-10-08
ORippler ORippler merged 71ad0590 into master 42 days ago

Login to write a write a comment.

Login via GitHub

Assignees
No one assigned
Labels
Milestone