onnxruntime
e927db3f - [WebGPU] Keep LayerNormalization arithmetic in FP32 (#32896)

Commit
12 days ago
[WebGPU] Keep LayerNormalization arithmetic in FP32 (#32896) ### Description Keep full LayerNormalization arithmetic in FP32 until the normalized value is converted to the output type. This prevents FP16 mean conversion from amplifying small FP32 rounding errors before subtraction, and applies to both ordinary and split normalization shaders. Preserve the existing same-type SimplifiedLayerNormalization arithmetic used by fused RMSNorm kernels. Mixed-type normalization already uses FP32. Input/output storage types and test tolerances are unchanged. ### Motivation and Context `LayerNorm_WebGPU.Float16` fails on a Radeon 880M with the D3D12 backend (driver `32.0.22024.3004`) for full normalization at shapes `{2,3}` and `{2,6}`, with and without bias. For `{2,3}`, the FP32 mean is `-1.7499998807907104`. Narrowing it toward zero gives `-1.7490234375` in FP16. Subtracting this from the middle input, `-1.75`, and normalizing produces `-0.004779815673828125` instead of zero, exceeding the existing `0.002` tolerance. The CPU oracle normalizes before converting to the output type. WGSL permits either adjacent representable value when narrowing to a floating-point type, so the shader cannot assume round-to-nearest will recover the exact mean. A standalone D3D12 probe reproduces the conversion behavior on both AMD and WARP; enabling IEEE strictness does not resolve it. The existing `LayerNorm_WebGPU.Float16` test provides regression coverage. Its related tests also cover mixed types, FP32, scalar/vec2/vec4, split normalization, bias and statistics outputs. No additional tolerance adjustment or duplicate test is needed. ### Validation - Clean Windows x64 Release build of `onnxruntime_provider_test` with WebGPU enabled (Ninja/MSVC), on the Radeon 880M machine above. - `--gtest_filter=LayerNorm_WebGPU.*`: all 5 tests pass, with no skips. The 4 numerical tests exercise 72 subcases with CPU fallback disabled, including 24 SimplifiedLayerNormalization subcases. - `--gtest_filter=LayerNormTest.*:CudaKernelTest.*LayerNorm*`: all 54 broader tests pass, with no skips, using `ORT_TEST_RANDOM_SEED_VALUE=2345`. These tests include CPU and WebGPU coverage as supported by each case. - The original `LayerNorm_WebGPU.Float16` failure was reproduced before the change. An independent D3D12 arithmetic probe also improves from 10 failing comparisons to zero across 36 outputs at the original tolerance. - Changed lines pass clang-format 20.1.8; `git diff --check` passes.
Author
Parents
Loading