llama.cpp
a01ce9ff - opencl: add several gemm variants for small batches and some fixes (#30850)

Commit
30 days ago
opencl: add several gemm variants for small batches and some fixes (#30850) * opencl: fix OOB read in `mul_mv_mxfp4_f32_flat` * opencl: remove large-N guard that forces CPU weight placement * this condition claims that large batch produces incorrect results on adreno and cannot be reproduced * model loader uses N512 to probe op support, resulting in cpu fallback * thus this condition does not seem necessary * opencl: cap num of subgroups by workgroup size for q4_0 mc3 Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> * opencl: add small batch variant for q4_k dp4a GEMM Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> * opencl: add a narrow variant of mul_mm_f16_f32_l4_lm Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> * opencl: add a one subgroup variant for gemv_noshuffle_q1_0_f32 for big M Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> --------- Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>
Author
Parents
Loading