onnxruntime
0b487785 - Bound windowed GQA workspace allocations (#32602)

Commit
58 days ago
Bound windowed GQA workspace allocations (#32602) ## Summary - Size sliding-window GroupQueryAttention Flash split workspace using the resident or staged KV extent instead of the unbounded absolute sequence length. - Size and launch the unfused fallback with the same effective KV length. - Preserve existing non-windowed behavior and all dispatch decisions. - Add host coverage for the effective-length calculation and CUDA regression coverage for `T >> C` on unfused and multi-token staged Flash routes. ## Stack Depends on #32454. The new change for this PR is commit `c9a579a6e8`; earlier commits belong to the existing GQA workspace-recipe stack. This is a runtime allocation-reduction prerequisite for the subsequent GQA Level-1/Level-2 workspace-estimation PR. It does not add planner opt-in, planned-pointer access, root slicing, or estimator wiring. ## Validation - CUDA provider test target built successfully. - CUDA internal module: 239 passed, 2 unrelated LeanAttention tests skipped. - Targeted GQA runtime tests: 5 passed, including windowed unfused `T=64, C=8` and staged Flash `T=64, C=8, S=3`. - Changed-file lintrunner passed. - `git diff --check` passed. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Author
Parents
Loading