whisper.cpp
c66c71e9 - vulkan: Use one row per workgroup for f32 mmv (llama/17711)

Commit
74 days ago
vulkan: Use one row per workgroup for f32 mmv (llama/17711) The MoE models have a mul_mat_vec with very small m (32, 64, 128) right before the topk_moe selection. Running multiple rows per wg doesn't utilize the SMs well. I think even for larger m, f32 is so bandwidth-limited that running multiple rows doesn't help.
Author
Committer
Parents
Loading