onnxruntime
bb47315d - [WebGPU] Widen MatMulNBits SubgroupMatrix-8x16x16 tiling to 128x256 (#32605)

Commit
5 days ago
[WebGPU] Widen MatMulNBits SubgroupMatrix-8x16x16 tiling to 128x256 (#32605) ### Description [WebGPU] Widen MatMulNBits SubgroupMatrix-8x16x16 tiling to 128x256 - Parameterize kTileM and kTileN in the GPU kernel to support dynamic tiling configurations based on workload size. - Increase the kernel tile size from 64x64 (256 threads), to 128x256 (512 threads) for larg M to improve performance. - Centralize the selected tiling policy into a SubgroupMatrixMatMulNBitsTiling struct. **Test on Phi-4-mini, Intel Panther Lake:** | | Base Prefill TPS | Opt Prefill TPS | Improve | |---|---|---|---| | prompt-128 | 1902.96 | 1838.73 | -3% | | prompt-256 | 2182.59 | 2391.95 | 10% | | prompt-512 | 2133.16 | 2370.76 | 11% | | prompt-1024 | 2111.78 | 2312.07 | 9% | | prompt-2048 | 1864.42 | 2173.03 | 17% | | prompt-4096 | 1447.15 | 1748.95 | 21% | ### Motivation and Context See above.
Author
Parents
Loading