onnxruntime
0656e947 - webgpu: add MatMulBnb4 contrib op support (#29587)

Commit
65 days ago
webgpu: add MatMulBnb4 contrib op support (#29587) Implement the MatMulBnb4 quantized matmul operator for the WebGPU execution provider, supporting both FP4 (quant_type=0) and NF4 (quant_type=1) blockwise 4-bit weight dequantization. ### Description <!-- Describe your changes. --> Add WebGPU execution provider support for the MatMulBnb4 contrib operator, enabling blockwise 4-bit quantized matmul (bitsandbytes-style FP4 and NF4) to run on WebGPU. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Models quantized with bitsandbytes 4-bit weights fell back to CPU when running on WebGPU, causing significant slowdowns, and cannot use graph capture feature. Adding a native WebGPU kernel keeps these quantized MatMulBnb4 on the GPU, avoiding costly EP fallback and CPU/GPU data transfers. marqo-fashionSigLIP-text-bnb4: ~30% faster marqo-fashionSigLIP-vision-bnb4: ~20% faster
Author
Parents
Loading