llama.cpp
b23701f7 - cuda : fix CUB argsort corruption caused by in-place keys (#28389)

Commit
9 days ago
cuda : fix CUB argsort corruption caused by in-place keys (#28389) argsort_f32_i32_cuda_cub called the one-shot DeviceRadixSort::SortPairs API with d_keys_in == d_keys_out (temp_keys, temp_keys). CUB's internal double-buffer ping-pong requires distinct key buffers: with aliased buffers the sort partially overwrites its own input mid-pass and emits a corrupted permutation, surfacing as intermittent garbage indices (e.g. backend top_k over a 248k-column vocab on Maxwell/CUDA 12.5/CCCL 2.x, which then triggered out-of-bounds gathers in downstream get_rows). Use a distinct keys-out buffer for all six call sites (plain and segmented, ascending and descending, size-query and execute). --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Oliver Simons <osimons@nvidia.com>
Parents
Loading