onnxruntime

b5375da8 - [WebGPU] Extend FP16 MatMul GEMV fast path to M=2–8 (#33351)

[WebGPU] Extend FP16 MatMul GEMV fast path to M=2–8 (#33351) ## Summary Extend the existing WebGPU FP16 MatMulGemv fast path from M=1 to 1 <= M <= 8, preserving its existing shape and structural constraints. Each row uses independent workgroups with the existing FP32 accumulation and reduction. The shader derives row and column-group indices from the flattened workgroup index and applies row offsets to input/output accesses. ## Motivation Small-M projections occur during speculative decoding. Previously, moving from one row to two rows selected generic MatMul, introducing a substantial performance cliff for skinny FP16 projections such as Qwen's recurrent gates: [M, 5120] x [5120, 48]. Isolated A100 / WebGPU / Vulkan measurements using Dawn GPU timestamps show: | M | Generic MatMul | Extended GEMV | Speedup | |---|---:|---:|---:| | 2 | 166 us | 30 us | 5.53x | | 4 | 167 us | 30 us | 5.57x | | 8 | 170 us | 33 us | 5.15x | These are median kernel timings from one warmup and three measured runs, not end-to-end generation results. ## Scope The existing eligibility conditions remain unchanged except for M: - FP16 inputs and output; single batch after existing batch folding. - 16 <= N <= 64, with N divisible by 4. - 2048 <= K <= 8192. - No bias or fused activation. Existing optimized-path precedence is preserved. No device-specific heuristic, MatMulNBits selection change, profiling infrastructure, or benchmark-only option is included. Regression coverage includes every M from 1 through 8, K/N boundaries, non-aligned K, folded rows, and fallback shapes outside the eligibility bounds. ## Validation of the original cleaned implementation - Rebuilt the production WebGPU runtime, Python binding, and provider tests. - 19 existing/new native tests passed; four existing unsupported cases skipped. - 25 GPU-only routing/reference checks passed: eligible M=2–8 cases selected GEMV, while excluded shapes retained their previous path. - M=1 matrix and vector outputs were bit-identical to the original runtime. - Formatting and git diff --check passed. GEMV retains the existing FP32 reduction. Newly eligible shapes can differ numerically from generic FP16 MatMul; bitwise equivalence is not claimed for M=2–8. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>