onnxruntime
f83bf57d - [CUDA] Plumb GQA workspace KV-length envelope knob into the L1 estimator (reader-only) (#33014)

Commit
30 days ago
[CUDA] Plumb GQA workspace KV-length envelope knob into the L1 estimator (reader-only) (#33014) ## What Adds the session option `ep.cuda.gqa_workspace_max_total_sequence_length` and threads its value into the CUDA GroupQueryAttention (GQA) Level-1 workspace estimator. `total_sequence_length` (accumulated past + current KV tokens) is a **runtime scalar input** whose value cannot be recovered from graph shapes. For the non-windowed KV cache, the backend workspace scales linearly with it, so the L1 estimator currently returns `std::nullopt` for that case (it cannot bound the value). This knob lets the user hand the estimator that **KV-length envelope** directly, for capacity-aware partitioning (#31962). ## No functional change (reader-only) This is a **pure NFC reader**: - The value is parsed (`WorkspaceEstimatorConfig` → `GQAWorkspaceEstimateConfig::max_total_sequence_length`) and **stored**, but **not consumed**. - `BuildBounds` still returns `std::nullopt` for the non-windowed case, so **the estimate output is unchanged** for all inputs. - A follow-up PR (PR 4) consumes the field to produce the non-windowed bound. The plumbing mirrors the existing `ep.cuda.fpa_intb_*` knobs (config key → `WorkspaceEstimatorConfig` → CUDA EP parse → estimator param). ## Test Extends the existing `RealAccountantTest.FactoryRetainsNarrowWorkspaceEstimatorConfig` framework test to validate the reader end-to-end (config string → `WorkspaceEstimatorConfig` field) in CPU CI. ## Context Part of the policy-based workspace-estimation PR chain (design doc: `docs/annotated_partitioning/policy_based_workspace_estimation.md`). This is **PR 3 (envelope plumbing)** in the roadmap: - PR 1 — GQA scratch sizing consolidation (#32945) - PR 2 — route-eligibility extraction (#33001) - **PR 3 — envelope plumbing (this PR)** - PR 4 — route-aware, envelope-bounded estimate (consumes this field) --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7fa98d07-aec3-41b7-92b8-b0ec317a95bf Copilot-Session: 37ad7066-fcd3-45df-891e-44bee9e1cf04
Author
Parents
Loading