llama.cpp
ggml : update CUDA/Metal radix Top-K to make tied selections deterministic
#31084

Merged

Commits

ggml : fix for the radix TOP_K thread timing bug

sanmai committed 7 days ago
make the key an ordered 64-bit pair

sanmai committed 7 days ago
a rewrite without shift-skip logic

sanmai committed 7 days ago
fix casts

sanmai committed 7 days ago
warp-contiguous layout

sanmai committed 7 days ago
reduce the scope removing vulkan+metal changes

sanmai committed 7 days ago
turn it off for Metal/ROCm, note about cub::DeviceTopK

sanmai committed 7 days ago
fix the hip build

sanmai committed 7 days ago
skip Vulkan as well

sanmai committed 7 days ago
popc lambda from fattn

sanmai committed 6 days ago
do block_cols math in int64_t

sanmai committed 6 days ago
make it allowlist bc SYCL looks non deterministic too

sanmai committed 6 days ago
ggml : make Metal radix Top-K tied selections deterministic

ggerganov committed 5 days ago
tests : enable deterministic top-k check on Metal, add radix-path tie cases

ggerganov committed 5 days ago
ggml : use coalesced accesses in the Metal radix Top-K compaction

ggerganov committed 5 days ago
tests : remove the extra top-k tie and perf cases

ggerganov committed 5 days ago
metal: scan the radix bins and warp totals with a simdgroup

ServeurpersoCom committed 5 days ago
tests: enable the deterministic top-k check on ROCm

ServeurpersoCom committed 5 days ago
cuda: remove the redundant radix top-k counter reset

ServeurpersoCom committed 5 days ago