onnxruntime
03a19a74 - [CPU/WebGPU] Support empty MatMulNBits weights (#33559)

Commit
1 day ago
[CPU/WebGPU] Support empty MatMulNBits weights (#33559) ### Description Support empty packed weights in CPU and WebGPU `MatMulNBits`. With `weight_prepacked=0`, one or more dimensions of B's packed shape may be zero; the other dimensions must retain their normal values. The output keeps the logical shape determined by A, K and N, and contains zeros or the broadcast bias. Unused quantization parameters may also be empty. CPU skips unused parameter packing and matrix multiplication; WebGPU records a fill dispatch so graph replay rewrites the output correctly. QKV/MLP fusion remains enabled: when a projection has empty weights, the fused operator uses the shared zero/bias fallback while computing the nonempty projections. Includes operator documentation and focused tests for FP16/FP32, 2/4/8-bit weights, constant/runtime inputs, bias, malformed shapes, empty/nonempty overrides, and graph capture. ### Motivation and Context Optional projections, such as disabled LoRA adapters, should not require full-size zero weight tensors. Zero-sized defaults let the same raw `MatMulNBits` graph run on CPU and WebGPU, with nonempty weights supplied when needed. Split from #33370. That PR retains these changes until this PR merges; QLoRA-specific operators, fusion and shaders remain there. ### Validation Built the standalone branch against main with built-in WebGPU on Windows x64. All 16 empty-weight tests, 28 existing QKV/MLP fusion regressions, and 8 existing nonempty-weight regressions passed; 4 Arm64-specific tests were skipped. Coverage includes fused empty/nonempty overrides, MLP bias, residual outputs, and capture replay. FP16 WebGPU cases require confirmed ShaderF16 support. Lint and `git diff --check` passed.
Author
Parents
Loading