llama.cpp
573aa89d - vulkan: Implement stream-k MUL_MAT (#28528)

Commit
1 day ago
vulkan: Implement stream-k MUL_MAT (#28528) * vulkan: Implement stream-k MUL_MAT Stream-k tries to split chunks of 256 K elements evenly across all SMs. Workgroups that finish a tile write to dst, others write to scratch, and there's a resolve shader similar to split-k. The shader code is implemented for scalar+cm1+cm2, but only currently enabled for cm2. The heuristic for enabling on coopmat2 is pretty simple due to the large tile size (one workgroup per SM), but would be more complex for the tile sizes being used for cm1, and it doesn't make sense for me to tune that on NVIDIA devices. Assisted-by: Codex * address review feedback
Author
Parents
Loading