onnxruntime
Add support for grouped-query MultiHeadAttention for CPU and CUDA
#31680
Open
Overview
Commits
19
Changes
View On GitHub

Add support for grouped-query MultiHeadAttention for CPU and CUDA #31680

copilot-swe-agent wants to merge 19 commits into main from copilot/add-grouped-query-support
copilot-swe-agent
copilot-swe-agent copilot-swe-agent assigned copilot-swe-agent copilot-swe-agent 60 days ago
copilot-swe-agent copilot-swe-agent assigned tianleiwu tianleiwu 60 days ago
azure-pipelines
tianleiwu tianleiwu changed the title Add CPU support for grouped-query MultiHeadAttention Add support for grouped-query MultiHeadAttention for CPU and CUDA 60 days ago
tianleiwu tianleiwu requested a review from titaiwangms titaiwangms 60 days ago
tianleiwu tianleiwu requested a review from copilot-pull-request-reviewer copilot-pull-request-reviewer 60 days ago
tianleiwu tianleiwu requested a review from apsonawane apsonawane 60 days ago
tianleiwu tianleiwu requested a review from kunal-vaishnavi kunal-vaishnavi 60 days ago
copilot-pull-request-reviewer
copilot-pull-request-reviewer commented on 2026-08-06
tianleiwu tianleiwu requested a review from hariharans29 hariharans29 52 days ago
tianleiwu tianleiwu force pushed from cddb69e2 to 785368c2 52 days ago
tianleiwu tianleiwu marked this pull request as ready for review 46 days ago
titaiwangms
titaiwangms commented on 2026-08-24
tianleiwu
titaiwangms
Copilot Add grouped-query MHA validation and CUDA path
fc7c88d8
Copilot Complete grouped-query MHA shapes and cache handling
d09280db
Copilot Finalize grouped-query MHA validation and tests
51dfb9e6
Copilot Implement CPU grouped-query attention
e5c0758a
Copilot Fix grouped-query shape inference build
88111b7f
tianleiwu address feedbacks
2b43d905
tianleiwu Address review feedback on grouped-query MultiHeadAttention
7688cfe8
tianleiwu Pass kv_num_heads explicitly to ApplyAttention
6da01191
tianleiwu Add hidden size accessors to AttentionParameters
c83a7809
tianleiwu Replace AttentionParameters::v_hidden_size with hidden size accessors
cbec80dd
tianleiwu fix build
887b17f7
tianleiwu consistent naming
54ba92b7
tianleiwu fix(webgpu): use grouped-query value dimensions
db4e5734
tianleiwu docs: regenerate contrib operator reference
aff244cc
tianleiwu docs: fix grouped-query attribute placement
555fb892
tianleiwu fix(attention): correct grouped-query caches
d1bd6a98
tianleiwu tianleiwu force pushed from c6831587 to d1bd6a98 33 days ago
tianleiwu Keep allowing bias when key and value are 4-D BNSH tensors
d574822b
tianleiwu tianleiwu force pushed from aa50ec59 to d574822b 31 days ago
tianleiwu tianleiwu requested a review from titaiwangms titaiwangms 31 days ago
titaiwangms
kunal-vaishnavi
kunal-vaishnavi commented on 2026-09-04
kunal-vaishnavi
kunal-vaishnavi commented on 2026-09-04
kunal-vaishnavi
kunal-vaishnavi dismissed these changes on 2026-09-04
kunal-vaishnavi
tianleiwu fix(attention): address GQA review feedback
83144876
tianleiwu tianleiwu dismissed their stale review via 83144876 30 days ago
tianleiwu
tianleiwu tianleiwu requested a review from kunal-vaishnavi kunal-vaishnavi 30 days ago
kunal-vaishnavi
kunal-vaishnavi commented on 2026-09-08
tianleiwu fix(attention): validate GQA head counts in symbolic shape inference
ced57815
titaiwangms titaiwangms requested a review from copilot-pull-request-reviewer copilot-pull-request-reviewer 19 days ago
copilot-pull-request-reviewer
copilot-pull-request-reviewer commented on 2026-09-15

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum