llama.cpp
f275595d - sycl: fix thread/block count in quantized cpy kernel launches (#27160)

Commit
9 hours ago
sycl: fix thread/block count in quantized cpy kernel launches (#27160) Adjusts the thread/block count to be proportional to the size of the quant, reducing under/over subscription. Largest perf improvement is the q4_0 -> f32 path, with, on a Arc 70, throughput goes from 20.21 GB/s to 158.19 GB/s The rest of the quants are flat in performance uplift.
Author
Parents
Loading