onnxruntime
baa0d32b - [WebGPU] Accumulate MatMulNBits wide-tile in output_element_t (#29611)

Commit
4 days ago
[WebGPU] Accumulate MatMulNBits wide-tile in output_element_t (#29611) ### Description Accumulate directly in output_element_t (e.g., f16 for f16 models) instead of hardcoding an f32 accumulator in the MatMulNBits wide-tile shader. **Intel Panther Lake** | | Prefill Length | Default Prefill TPS | Optimized Prefill TPS | Improvement | | :--- | ---: | ---: | ---: | ---: | | gpt-oss-20b-ONNX | 128 | 305.70 | 344.49 | 113% | | gpt-oss-20b-ONNX | 1024 | 396.50 | 429.80 | 108% | | Phi-4-mini-instruct-ONNX | 128 | 515.90 | 592.36 | 115% | | Phi-4-mini-instruct-ONNX | 1024 | 615.39 | 753.40 | 122% | [1] https://huggingface.co/onnx-community/gpt-oss-20b-ONNX [2] https://huggingface.co/onnx-community/Phi-4-mini-instruct-ONNX ### Motivation and Context See above.
Author
Parents
Loading