onnxruntime
f33c1744 - [CUDA][WebGPU] Add PackedSparseAttentionIndexerMerge for MTP index sharing (#33224)

Commit
31 days ago
[CUDA][WebGPU] Add PackedSparseAttentionIndexerMerge for MTP index sharing (#33224) ### Description Introduce `PackedSparseAttentionIndexerMerge` and extend `PackedSparseAttentionIndexer` to support index sharing, configurable output capacity, and explicit state-sharing policies. - Add the merge operator for combining base and additional sparse-attention indices, with CUDA and WebGPU support and dedicated documentation. - Extend input/output slot definitions, operator schemas, and kernel registration for shared indices and merged indices, counts, and status, including in-place merged-tensor operations. - Update the CUDA indexer to handle `indexshare_mode`, validate interface consistency, and manage merged outputs. - Add `max_output_entries` to control output capacity consistently across output shapes and QSA/CSA kernel parameters. - Add `indexshare_state_policy` to control CSA state advancement and sharing, with validation and error handling for incompatible interface usage. ### Motivation and Context Advanced sparse-attention workflows need to reuse selected indices and combine them with additional selections rather than generate each index set independently. These changes provide explicit index-sharing and merging interfaces to support those workflows. Configurable output capacity lets callers bound selection outputs, while an explicit CSA state policy distinguishes state advancement from sharing. Together, these controls make index reuse more flexible and state handling more predictable in large-scale transformer models. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Parents
Loading