sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (#25063)
* sycl: add supported types to ggml_sycl_supports_reorder_dmmv
The reordered feature is implemented in ggml_sycl_op_dequantize_mul_mat_vec,
but gated by ggml_sycl_supports_reorder_dmmv. This commit fixes the gate.
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* sycl: set K_QUANTS_PER_ITERATION=1 to improve utilization
When combined with opening the reorder gate, this improves GPU
utilization on B70, giving a significant boost to tg t/s.
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* sycl: replace QK_WARP_SIZE with WARP_SIZE for QK_5
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* sycl: add missing types to ggml_backend_sycl_buffer_init_tensor
Without this, the extra field is not allocated and the reorder path
will not take effect.
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
---------
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
