onnxruntime
ec1d573c - Add WebGPU INT2 and mixed-width QMoE support (#32878)

Commit
23 days ago
Add WebGPU INT2 and mixed-width QMoE support (#32878) ## Description Adds WebGPU execution support for INT2 and mixed-width integer `QMoE`, following the operator contract introduced for CUDA in #32761. - Accepts 2-bit expert weights on WebGPU. - Passes FC1, FC2, and FC3-specific bit widths to the existing `MatMulNBits` implementation. - Replaces the mixed-width rejection test with a nonzero INT2 FC1 / INT4 FC2 numerical test for the optimized single-token path. The underlying WebGPU `MatMulNBits` shaders already implement 2-bit unpacking, so no new shader path is required. ## Validation - [x] `clang-format --dry-run --Werror` on changed C++ files - [x] `git diff --check` - [x] VS Code diagnostics clean - [x] Release `onnxruntime_provider_test` build with WebGPU/D3D12 - [x] `QMoETest_WebGPU_MixedWidth_SingleToken` in hardware-backed WebGPU CI - [x] All WebGPU build/test CI jobs passed Local virtual D3D12 execution is unstable with `DXGI_ERROR_DEVICE_REMOVED`; hardware-backed CI is the numerical validation source. ## Review follow-up (October 5) Added H200/Vulkan coverage for homogeneous inherited INT2, k=2 mixed-width indirect selection of two distinguishable experts, two-token mixed-width routing to different experts, and separate non-fused SwiGLU FC3 with an independent INT2 width. FC1, FC2, and optional FC3 now reject packed rows/blocks that cannot be indexed as u32 words; tests cover the 12-element INT2 boundary and FC3. All 16 focused WebGPU QMoE tests pass locally; all six review threads have replies and are resolved. CI is rerunning on the new head. The previous Android minimal binary-size check exceeded its 1,597,440-byte threshold by 509 bytes; this is not claimed fixed by the WebGPU-only changes and must be reassessed on the rerun. --------- Co-authored-by: Sayan Shaw <sayanshaw@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Author
Parents
Loading