onnxruntime

6df2a551 - [CUDA] Add win-arm64 packaging and size options to plugin pipeline (#31635)

[CUDA] Add win-arm64 packaging and size options to plugin pipeline (#31635) ## Description Extends the CUDA plugin EP packaging pipeline with Windows ARM64 support, replaces the shared x64/ARM64 architecture lists with per-platform lists that reflect the GPUs each package actually targets, and adds two build-time knobs for controlling binary size. The plugin `.so`/`.dll` is currently very large (~720 MB uncompressed on the CUDA 12.8 Linux leg, ~90% of which is `.nv_fatbin`), and the architecture list is the single biggest lever on that number, so it needs to be tuned per platform rather than shared. ## Summary of Changes ### Windows ARM64 packaging | File | Change | |------|--------| | `plugin-cuda-pipeline.yml` | Add `build_windows_arm64`; rename `invalidAArch64Config` to `invalidArm64Config` and extend it to cover Windows ARM64, since NVIDIA only ships Windows-on-ARM CUDA for 13.x | | `plugin-cuda-packaging-stage.yml` | Add `build_windows_arm64` and `arm64_cuda_version` (13.1), wire the ARM64 stage and its artifacts into NuGet and Foundry Local zip packaging | | `plugin-win-cuda-stage.yml` | Add an `arm64` arch path: ARM64 agent pool, `win-arm64/` CUDA SDK blob prefix, separate cuDNN folder, native ARM64 toolset | ### Per-platform CUDA architecture lists `cmake_x64_cuda_archs` / `cmake_arm64_cuda_archs` are split into four independent lists, since Windows x64, Linux x64, Windows ARM64, and Linux aarch64 serve very different GPU populations: | Parameter | CUDA 12.8 | CUDA 13.x | |------|------|------| | `cmake_windows_x64_cuda_archs` | `61,75,86,89,120` | `75,80,86,89,120` | | `cmake_windows_arm64_cuda_archs` | n/a | `120,121` | | `cmake_linux_x64_cuda_archs` | `75,80,86,89,90,120` | `75,80,86,89,90,120` | | `cmake_linux_aarch64_cuda_archs` | n/a | `89,90,100,103,120,121` | Notable decisions: - **`120-virtual` dropped everywhere.** The `compute_120` PTX measured 176 MB, 27% of the entire `.nv_fatbin` — by far the most expensive single entry. It also cannot carry the NVFP4 kernels, which are only valid as real `sm_120a` (`cuobjdump -ptx | grep -c e2m1x2` returns 0), so it was paying full price for partial coverage. - **Linux aarch64 targets the platforms that actually exist on ARM**: GH200 (`90`), GB200 (`100`), GB300 (`103`), DGX Spark GB10 (`121`), plus discrete cards in ARM chassis (`89`, `120`). `103` is required alongside `100` because ORT normalizes `100` to `100a-real`, and `a` targets are locked to their exact SM. - **`75` dropped from Linux aarch64** — Turing was never paired with an ARM host in practice. ### CUDA architecture normalization | File | Change | |------|--------| | `cmake/external/cuda_configuration.cmake` | `ARCHITECTURES_WITH_ACCEL`: add `103` and `121`, drop `101` (removed by NVIDIA after CUDA 12.9). Without this, `103` and `121` would be built as plain targets and would silently lose the CUTLASS block-scaled/TMA kernels, which are gated on `__CUDA_ARCH_FEAT_SM1xx_ALL`. | ### Build size controls Two new pipeline parameters, both plumbed through the packaging stage to all four platform stages: | Parameter | Default | Effect | |------|------|------| | `enable_cuda_fatbin_size_compression` | `false` | Sets the new `onnxruntime_CUDA_FATBIN_COMPRESS_SIZE` cmake option, forcing `-Xfatbin=-compress-all -compress-mode=size` on the CUDA 12.8 leg. CUDA >= 13.0 already does this unconditionally, so the parameter only changes 12.8. | | `enable_fpa_intb_gemm` | `true` | Sets the existing `onnxruntime_USE_FPA_INTB_GEMM` cmake option. fpA_intB GEMV/GEMM is ~141 MB of device code (22% of `.nv_fatbin`), second only to flash attention. | `onnxruntime_CUDA_FATBIN_COMPRESS_SIZE` fails configuration on CUDA < 12.8 rather than silently passing an unsupported flag to nvcc. Windows composes these via `FatbinCompressOption` / `FpaIntBGemmOption` job variables appended to the `build.py` invocations, mirroring the existing `$(TelemetryOption)` pattern. Linux composes them into `EXTRA_CMAKE_DEFINES`, which `build_cuda_plugin_package.sh` already forwards. ### Packaged binary hardening and verification | File | Change | |------|--------| | `cmake/onnxruntime_providers_cuda_plugin.cmake` | Compile `onnxruntime_providers_cuda.rc` into the plugin DLL on Windows so the packaged binary carries version info; set `SKIP_BUILD_RPATH` on Linux so the build machine's CUDA path is not embedded in a binary that ships as-is | | `plugin-linux-cuda-stage.yml` | Fail the build if the plugin `.so` has an empty `RPATH`/`RUNPATH` component or a hard-coded CUDA path | | `plugin-win-cuda-stage.yml` | Fail the build if the plugin DLL is missing required version-info properties | ## Testing - Pipeline changes are validated by running the CUDA plugin packaging pipeline. Both new parameters default to current behavior (`enable_cuda_fatbin_size_compression: false`, `enable_fpa_intb_gemm: true`), so a default run produces the same build flags as before this PR aside from the architecture list changes. - The cmake `-compress-mode` selection logic was verified in isolation across four combinations: | Toolkit | Option | Result | |---|---|---| | 12.8 | OFF | `-Xfatbin=-compress-all` | | 12.8 | ON | `-Xfatbin=-compress-all -compress-mode=size` | | 13.1 | OFF | `-Xfatbin=-compress-all -compress-mode=size` | | 12.6 | ON | configure-time fatal error, as designed | - The new RPATH and DLL version-info checks are self-verifying: they fail the packaging stage rather than publishing a bad artifact. ## Motivation and Context The primary consumer is Foundry Local (vision, audio, and mostly LLM models), which ships this plugin to end-user machines, so download size matters directly. Trade-offs worth flagging for reviewers: - **`-compress-mode=size` raises the minimum driver** to the CUDA 12.4 level (Linux >= 550.54.14, Windows >= 551.61); older drivers cannot decompress the fatbin at all. It also increases module load time (measured ~0.8 ms to ~4.3 ms for a ~6.5 MB SASS module) and adds a few percent to nvcc time. This is why the parameter defaults to `false` and is opt-in per run. - **The architecture lists are all `-real` with no virtual entry**, so any GPU whose compute capability is not explicitly listed gets `cudaErrorNoKernelImageForDevice` (209) instead of falling back to JIT. This is deliberate given the PTX cost, but it means new architectures must be added explicitly. - **`enable_fpa_intb_gemm: false` is not yet validated end to end.** The fpA_intB path is opt-in at run time via `ORT_FPA_INTB_GEMM` / `ep.cuda.fpa_intb_gemm`, but `matmul_nbits.cc` forces it on whenever weights are prepacked, independent of that flag. The fallback path should be exercised before shipping a package built with this off. ## Checklist - [x] No breaking changes to default pipeline behavior (both new parameters default to existing behavior) - [x] cmake option gated on toolkit version with an explicit error rather than a silent no-op - [ ] Tests added/updated — not applicable; changes are build/packaging configuration