onnxruntime

74810e56 - [CUDA] Fix packed-INT QMoE expert-counting capture and consume (#32942)

[CUDA] Fix packed-INT QMoE expert-counting capture and consume (#32942) ### Description <!-- Describe your changes. --> Fix the CUDA QMoE packed-INT GEMV path's remaining reference to the removed `routing_record` API and add a focused expert-counting regression. - Capture selected expert IDs immediately after routing and consume the snapshot before the packed-INT path returns, using the same lifecycle as the neighboring paths. Preserve the existing non-plugin/non-minimal guards. - Add `MoeExpertCountingTest.CudaQMoEPackedIntGemv`: FP16, INT2, fused SwiGLU, three rows, four experts, top-k 1, 512-wide hidden/intermediate dimensions, and block size 64. - Verify expert-dependent numerical outputs and per-invocation selection counters: duplicate routes `[0, 0, 2]`, a repeated invocation, then changed routes `[1, 1, 1]`. Expected counters are `[0.1, 0, 0.1, 0]`, `[0.19, 0, 0.19, 0]`, and `[0.171, 0.1, 0.171, 0]`. - Disable CPU fallback and cap dense dequantization scratch at one byte, so a successful run cannot silently use the dense fallback. Only `moe_quantization.cc` and `moe_expert_counting_test.cc` change. Routing selection and GEMV arithmetic are unchanged; no WebGPU/PagedAttention changes are included. **Validation** Built and executed locally on NVIDIA A100-SXM4-80GB (SM80), CUDA 12.8.93, GCC 11.4, cuDNN 9.10.2.21, Release, warnings-as-errors, QUICK_BUILD=OFF. The existing CUDA build tree was reconfigured from SM86 to SM80, recompiling the CUDA targets. This is not a claim of CUDA 13/GCC 14 CI validation. After configuring with the same options plus `--update`, the following build command passed: ```bash source /workspace/venv/bin/activate python tools/ci_build/build.py \ --build_dir /workspace/onnxruntime/build/cuda-ci-20260929 \ --config Release --build --parallel 16 --cmake_generator Ninja \ --build_shared_lib --build_wheel --enable_onnx_tests --use_cuda \ --use_binskim_compliant_compile_flags --nvcc_threads 4 --flash_nvcc_threads 4 \ --cuda_version=12.8 --cuda_home=/usr/local/cuda-12.8 \ --cudnn_home=/workspace/venv/lib/python3.11/site-packages/nvidia/cudnn \ --enable_cuda_profiling --skip_tests --allow_running_as_root \ --targets onnxruntime_test_all onnxruntime_provider_test \ --cmake_extra_defines CMAKE_CUDA_ARCHITECTURES=80 \ onnxruntime_BUILD_UNIT_TESTS=ON \ onnxruntime_ENABLE_CUDA_EP_INTERNAL_TESTS=ON \ onnxruntime_BUILD_CUDA_QUANT_PREPROCESS=ON ``` Tests were run from the build output directory: ```bash cd /workspace/onnxruntime/build/cuda-ci-20260929/Release export LD_LIBRARY_PATH=/workspace/venv/lib/python3.11/site-packages/nvidia/cudnn/lib:/usr/local/cuda-12.8/lib64:${LD_LIBRARY_PATH:-} ./onnxruntime_test_all --gtest_filter='MoeExpertCountingTest.CudaQMoE*' # PASS: 4 tests, no skips. ./onnxruntime_provider_test --gtest_filter='MoETest.QMoETest_Int2CudaPackedDecodeMultiRowExpertParity' # PASS: 1 test, no skips. ORT_ENABLE_QMOE_KERNEL_DEBUG_INFO=1 ./onnxruntime_test_all \ --gtest_filter='MoeExpertCountingTest.CudaQMoEPackedIntGemv' # PASS: 1 test; all three invocations log Route=packed_int_gemv Rows=3. ``` **Negative control:** temporarily omitted both packed-path `Capture` and `Consume`, rebuilt with `cmake --build build/cuda-ci-20260929/Release --target onnxruntime_providers_cuda --parallel 16`, and ran `./onnxruntime_test_all --gtest_filter='MoeExpertCountingTest.CudaQMoEPackedIntGemv'`. It failed as intended: outputs still matched, but seven counter assertions failed because all counters remained zero (first failure: expert 0 expected 0.1, actual 0). Restored the repair, rebuilt the provider, and reran both focused suites: all five tests passed. No negative-control edits are committed. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> The expert-counting migration in #32738 left a stale `routing_record->CaptureTile` call in the packed-INT GEMV early-return path. This prevents ordinary built-in CUDA compilation on current main; simply deleting that call would compile but silently omit expert counts for this path. This separates the repair originally included in #32808, following [the scope/coverage review comment](https://github.com/microsoft/onnxruntime/pull/32808#discussion_r4137574578). The production patch is carried unchanged from commit `1f57e92abe`; the new regression verifies the runtime counting behavior as well as compilation. After this lands, merging main into #32808 will remove the unrelated MoE file from that PR's diff. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>