[CUDA] Add persistent fpA_intB MatMulNBits tactic autotune cache (#29588)
### Description
Adds an **opt-in, hardware-keyed on-disk tactic cache** for the fpA_intB
weight-only `MatMulNBits` path, plus a reduced first-time profiling
sweep with lazy fallback. Persistence is off by default: with no cache
location configured, behavior is unchanged (in-process only).
**Cache subsystem** (`gemm_tactic_cache.{h,cc}`)
- `HardwareSignature` (GPU name, SM, CUDA runtime, ORT version;
commit/build config/driver/MP count recorded for diagnostics) names and
guards cache files, so tactics are never reused across incompatible
GPUs/toolkits/ORT versions. A strict `tactic_selection_version` header
field rejects caches tuned under an older selection policy, even when
the ORT version is unchanged.
- TSV format: percent-encoded free-text fields, a column-name header
(appended columns are backward/forward compatible), file locking +
atomic write through a per-writer temp file and rename (`MoveFileExW` on
Windows; paths are UTF-8), in-memory merge on flush so concurrent
sessions tuning different shapes don't lose updates, and creation of a
missing cache directory on first write.
- `MatMulNBitsTacticCache` keyed by `(N16b, K, act dtype, weight type,
bits, block_size, zero-points, gemv_enabled, packing_sm)`. `packing_sm`
keeps the SM80-compat and native-SM90 tactics for the same shape
distinct on one device (e.g. H200). 2-, 4-, and 8-bit weights are keyed
separately.
**Profiler** (`gemm_profiler.h`, `fpA_intB_gemm_profiler.{h,cc}`)
- Reduced initial sweep to a small M bucket set (`{1,2,4,…,2048}`
default; override with `ep.cuda.fpa_intb_profile_m` /
`ORT_FPA_INTB_PROFILE_M`). `getBestConfigOrProfile` lazily profiles any
missing bucket on first use.
- Persistent-cache load/store hooks are no-ops unless a cache is
attached.
- CUTLASS tactics loaded from disk are validated against the runner's
current `getConfigs()` and dropped (re-profiled) if incompatible.
**Wiring** (`matmul_nbits.{h,cc}`, `cuda_execution_provider.cc`, plugin
`cuda_ep.cc`)
- Cache location: session config (`ep.cuda.gemm_tactic_cache_prefix` /
`_dir`) takes precedence over env (`ORT_CUDA_GEMM_TACTIC_CACHE_PREFIX` /
`_DIR`); a prefix wins over a dir.
- Process-global caches are keyed by resolved file path; the file is
loaded once per location (not once per node), and the signature is built
from the kernel's cached device properties.
- The process-global in-memory profile map (`GemmIdCore`) is keyed by
the same quantization fields as the persistent key (bits, block size,
zero points, GEMV support), so kernels with different quantization never
share tactics.
- Construction-time sweeps and lazily profiled buckets are staged in
memory as soon as they are profiled, and all disk writes are batched at
CUDA EP teardown (`~CUDAExecutionProvider`, or `~CudaEp` for the plugin
EP), so there is no disk I/O on the inference path.
- Lazy profiling is skipped while the compute stream is capturing a CUDA
graph; run a warmup inference before capture.
**Offline tuning tool**: `onnxruntime/python/tools/fpa_intb_tune.py`
(uses session config entries; releases the session before summarizing so
lazily tuned buckets are flushed).
### Testing
- `gemm_tactic_cache_test.cc` (built into
`onnxruntime_providers_cuda_ut` with
`onnxruntime_ENABLE_CUDA_EP_INTERNAL_TESTS=ON`, run via
`--gtest_filter=CUDA_EP_Unittest.All`): TSV round-trip and malformed
escapes, SM80/SM90 config serialize/parse, null tactic, store/load,
signature-mismatch rejection, non-strict commit/build config,
selection-version rejection, appended-column tolerance, concurrent
flush-merge, missing-directory creation, file path resolution, and
`GemmIdCore` separation of quantization variants.
- `*MatMulNBits*` CUDA integration tests.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>