onnxruntime

c4f76f6f - Detach C++ node tests from ONNX on-disk data/node artifacts (onnx#7959) + retiring equivalence proof + corpus-collapse tripwire (#29504)

Detach C++ node tests from ONNX on-disk data/node artifacts (onnx#7959) + retiring equivalence proof + corpus-collapse tripwire (#29504) ### Motivation ONNX onnx/onnx#7959 removes the on-disk `onnx/backend/test/data/node/` corpus (targeted for ONNX 1.23), replacing it with on-the-fly in-memory generation. ORT's C++ `onnx_test_runner` reads that corpus from disk — after the deletion it would load **0 node tests and silently exit 0** (green CI on a corpus that no longer exists). This PR detaches ORT from the deleted artifacts *ahead* of the bump. ### What this does (3 pieces) 1. **Detach** — a build-time materializer (`tools/python/materialize_onnx_node_tests.py`) regenerates the node corpus to disk from ONNX's surviving generator (`collect_testcases`), so the C++ runner keeps reading from disk unchanged. EP-agnostic (CPU/CUDA/QNN share one materialized tree). 2. **Equivalence proof** — a *retiring* test (`onnxruntime/test/python/onnx_node_test_equivalence_test.py` + `tools/python/compare_node_test_corpora.py`) proves the materialized corpus is byte-identical to the original (modulo a documented ULP band); it auto-skips once the ONNX on-disk oracle disappears post-#7959. 3. **Cause-agnostic tripwire** — build-time `--min-cases` gate (FATALs the build on every MATERIALIZE=ON leg if generation < floor) + a runtime `-m` floor on the CPU node ctest, turning silent-green-on-empty into a loud red. ### Key decisions - **numpy**: cmake configure = HARD FATAL on off-pin numpy (build-env reproducibility across CI legs); Python materializer = SOFT WARN (standalone advisory). Different layers, different questions. The onnx pin stays HARD on both sides. - **QNN legs**: repointed to the materialized tree + pinned onnx/numpy installed pre-build. Node-dir runs carry a **low `-m 1` collapse sentinel** (NOT the 1500 build floor): `-e qnn` legitimately reduces the collected set to ~1529, so a 1500 runtime floor would be a false-red timebomb on the next opset bump — the strict count is enforced at build time via `--min-cases`, while `-m 1` (zero false-red risk) catches a per-leg `MATERIALIZE=OFF` that still runs the node dir. The android leg's single-case `cp -r` source is repointed to the materialized tree (the old `data/node` source vanishes post-#7959). ### Testing - Runtime tripwire empirically verified: empty/truncated corpus → runner FATAL (nonzero); full → exit 0; default (no `-m`) unchanged. - Equivalence: 1799-case byte-probe (dir-set match; byte-identical modulo the documented ULP band). - cmake configure verified to generate cleanly on latest main (ep_context + node-test regions coexist, no collisions). - **[needs-run]** on a pinned-numpy CI box: the `MATERIALIZE=ON` inner path (actual materialization + the two add_test bodies) — this dev box is off-pin so the numpy gate FATALs by design. Relates to: onnx/onnx#7959 --- ## Cross-consumer viability note (re: onnx/onnx#7959) While detaching, we assessed all of ORT's node-test consumers. The in-memory generator approach behind #7959 works cleanly for **single-version** consumers (C++, C#, docs), but has a gap for consumers needing a **historical multi-opset matrix**: - ORT's JS/web tests pull node data for opset 7–21 from 15 immutable `rel-*` release archives. Since onnx's generator is single-version, old opsets can't be regenerated from a new onnx. - Those consumers stay green post-#7959 only because released branches are immutable — there is no generator-based path to add a *new* post-#7959 opset for them. Net: workable for single-version consumers; the multi-version case would need a small per-version serialization utility or a documented migration note upstream. --- --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>